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PREFACE 


This  report  was  conducted  as  part  of  the  Evaluation  of  Environmental  Investments  Research  Program 
(EEIRP).  The  EEIRP  is  sponsored  by  Headquarters,  U.S.  Army  Corps  of  Engineers  (HQUSACE).  It  is  jointly 
assigned  to  the  U.S.  Army  Engineer  Water  Resources  Support  Center  (WRSC),  Institute  for  Water  Resources 
(IWR),  and  the  U.S.  Army  Engineer  Waterways  Experiment  Station  (WES),  Environmental  Lab  (EL).  Mr. 
William  J.  Hansen  of  IWR  is  the  Program  Manager,  and  Mr.  H.  Roger  Hamilton  is  the  WES  Manager. 
Program  Monitors  during  this  study  were  Mr.  John  W.  Bellinger  and  Mr.  K.  Brad  Fowler,  HQUSACE.  The 
field  review  group  members  that  provide  complete  Program  direction  and  their  District  or  Division  affiliations 
are  Mr.  David  Carney,  New  Orleans  District;  Mr.  Larry  Kilgo,  Lower  Mississippi  Valley  Division;  Mr. 
Richard  Gorton,  Omaha  District;  Mr.  Bruce  D.  Carlson,  St.  Paul  District;  Mr.  Glendon  L.  Coffee,  Mobile 
District;  Ms.  Susan  E.  Durden,  Savannah  District;  Mr.  Scott  Miner,  San  Francisco  District;  Mr.  Robert  F. 
Scott,  Fort  Worth  District;  Mr.  Clifford  J.  Kidd,  Baltimore  District;  Mr.  Edwin  J.  Woodruff,  North  Pacific 
Division;  and  Dr.  Michael  Passmore,  WES  EL.  The  work  was  conducted  under  the  Incorporating  Risk  and 
Uncertainty  into  Environmental  Evaluation  Work  Unit  of  the  EEIRP.  Mr.  L.  Leigh  Skaggs  of  the  Technical 
Analysis  and  Research  Division  (TARD),  IWR  and  Mr.  Richard  Kasul  of  the  Natural  Resources  Division 
(NRD),  WES  are  the  Principal  Investigators. 

The  work  was  performed  by  the  Greeley  Polhemus  Group,  Inc.  (GPG)  under  Task  Order  No.  28, 
Contract  No.  DACW-72-90-D-0002,  managed  by  Mr.  Leigh  Skaggs.  Dr.  Charles  Yoe  was  the  principal 
investigator.  The  report  was  edited  by  Mr.  Leigh  Skaggs. 

Valuable  review  comments  and  suggestions  were  received  from  Mr.  John  Bellinger,  HQUSACE;  Mr. 
Jim  Henderson,  WES  EL;  Mr.  Richard  Kasul,  WES-EL;  Mr.  William  Hansen,  IWR;  Dr.  David  Moser,  IWR; 
Mr.  Bill  Werick,  IWR;  and  Dr.  Justin  Williams,  IWR. 

The  report  was  prepared  under  the  general  supervision  at  IWR  of  Mr.  Michael  R.  Krouse,  Chief, 
TARD;  and  Mr.  Kyle  E.  Schilling,  Director,  IWR;  and  at  EL  of  Dr.  Robert  M.  Engler,  Chief,  NRD  and  Dr. 
John  W.  Keeley,  Director,  EL. 

At  the  time  of  the  preparation  of  this  report,  Mr.  Kyle  Schilling  was  Acting  Director  of  WRSC  and 
Dr.  Robert  W.  Whalin  was  Director  of  WES.  Commander  of  WES  was  COL  Bruce  K.  Howard,  EN. 
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EXECUTIVE  SUMMARY 

This  report  was  prepared  to  provide  assistance  to  the  U.S.  Army  Corps  of  Engineers  (Corps)  in 
beginning  to  assess  risk  and  uncertainty  in  the  evaluation  of  environmental  investments.  The  literature 
review  presented  in  this  report  was  developed  as  an  initial  effort  for  the  “Incorporating  Risk  and  Uncertainty 
into  Environmental  Evaluation”  work  unit  of  the  Corps’  Evaluation  of  Environmental  Investments 
Research  Program  (EEIRP).  The  products  of  the  EEIRP  provide  Corps  planners  with  methodologies  and 
techniques  to  aid  in  developing  supportable  environmental  restoration  and  mitigation  projects  and  plans. 

This  literature  review  is  primarily  for  Corps  personnel,  but  it  has  two  audiences.  It  serves  as  both 
a  primer  on  the  generic  risk  analysis  literature  that  will  help  planners  find  the  tools  they  need  to  do  risk 
analysis,  as  well  as  a  gateway  to  help  more  experienced  personnel  find  their  way  into  the  environmental 
resource  risk  analysis  literature. 

The  first  task  in  this  research  was  to  review  the  Corps'  existing  risk  analysis  guidance  to  determine 
how  it  may  be  useful  in  crafting  the  future  development  of  risk  and  uncertainty  analysis  guidelines  for 
evaluating  environmental  investments.  The  review  begins  with  an  introduction  to  the  concept  of  ecosystem 
restoration.  To  provide  a  contextual  approach  to  the  review,  the  generic  notion  of  uncertainty  is  considered 
in  the  ecosystem  restoration  planning  process.  A  taxonomy  of  uncertainty  is  offered  as  a  starting  point  for 
understanding  the  types  and  sources  of  uncertainty  encountered  in  environmental  resources  planning. 

The  context  begins  with  a  review  of  two  different  ways  to  look  at  probabilities.  Next,  there  is  a 
generic  discussion  of  the  types  of  quantities  encountered  in  ecosystem  restoration  planning  that  can  be 
uncertain.  The  different  types  of  quantities  that  are  uncertain  can  have  different  sources  of  uncertainty,  so 
a  discussion  of  the  sources  of  uncertainty  follows.  The  contextual  information  looks  at  possible  sources  of 
uncertainty  that  may  be  encountered  in  an  ecosystem  restoration  planning  framework.  The  remainder  of 
Chapter  Two  comprises  the  review  of  specific  Corps'  risk  analysis  guidance. 

Existing  Corps  guidance  for  risk  analysis  tends  to  consist  of  two  types:  1)  general  concepts, 
principles,  and  tools  as  discussed  in  manuals  and  reports;  and  2)  detailed  and  specific  guidance  as  presented 
in  Engineer  Circulars  (EC’s)  and  Engineer  Technical  Letters  (ETL’s).  There  is  much  in  the  former  guidance 
that  can  be  readily  incorporated  into  environmental  risk  analysis  guidance.  There  is  much  in  the  latter 
guidance  to  be  emulated  by  environmental  risk  analysis,  but  very  little  that  can  be  directly  adapted.  Sample 
analyses,  literature  reviews,  lists  of  key  variables,  introductions  to  the  array  of  techniques  and  toots  available 
for  risk  and  uncertainty  analyses,  and  models  to  be  followed  in  structuring  any  potential  future  environmental 
risk  analysis  guidelines  are  among  the  most  useful  things  to  be  gleaned  from  the  Corps’  existing  guidance. 

Chapter  Three  contains  a  review  of  the  professional  literature.  Fifty-two  books,  reports,  papers  or 
articles  have  been  reviewed  at  length  for  this  report.  They  are  presented  in  two  sections,  following  an 
introduction  that  describes  the  review  process.  The  first  section  contains  literature  that  addresses  the  general 
methodology  of  risk  and  uncertainty  analysis.  It  introduces  readers  to  the  literature  that  will  help  them  find 
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access  to  the  general  tools  and  techniques  of  risk  and  uncertainty  analysis.  The  emphasis  here  is  on 
introductory  level  readings,  though  not  to  the  exclusion  of  some  intermediate  and  advanced  sources  of 
information.  The  second  section  of  the  professional  literature  review  includes  applications  of  risk  analysis 
to  environmental  issues. 

Each  annotation  begins  with  a  complete  citation  of  the  book,  report,  paper  or  article.  There  follows 
a  section  called  “applicability”,  which  is  intended  to  help  the  reader  decide  whether  they  are  interested  in 
reading  the  entire  review.  Following  is  the  “review”  portion  of  the  summary,  where  the  content  of  the  item 
is  summarized. 

A  useful  summary  of  the  52  books  and  articles  is  presented  in  Table  3-1.  For  each  annotated  item, 
the  following  information  is  provided:  title;  year  of  publication;  name  of  first  author;  whether  the  item  is 
a  book,  a  paper  from  a  journal,  a  government  report,  or  an  article  from  a  collection  of  writings;  the  type  of 
reader'who  is  likely  to  find  the  article  most  helpful  or  interesting  (planners,  analysts,  mangers,  novices, 
experts);  and  the  three  most  important  topics  or  ideas  addressed  in  the  item. 

In  addition  to  the  52  summaries,  there  are  two  more  sections  of  the  professional  literature  review. 
The  third  section  provides  some  suggestions  to  the  reader  for  sources  of  information  worth  monitoring  for 
■future  developments  in  the  literature.  The  final  section  of  the  professional  literature  provides  an  extended 
traditional  bibliography  of  books  and  articles  of  potential  interest. 

The  authors  arrived  at  several  observations  in  the  course  of  conducting  the  literature  review.  One 
of  the  most  significant  observations  is  that  the  language  of  risk  analysis  has  many  meanings.  Titles  of  books 
and  papers,  annotations,  lists  of  keywords,  and  the  contents  of  much  that  is  written  indicate  a  striking  lack 
of  uniformity  in  the  language  of  risk  analysis.  For  example,  most  of  the  numerous  references  to 
“environmental  risk  analysis”  refer  to  the  quantitative  risk  assessment  of  human  health  effects  of  projects 
and  actions,  a  key  interest  of  the  Environmental  Protection  Agency  (EPA).  Human  health  or  environmental 
risk  assessment  estimates  the  incidence  of  death  and  disease  to  humans  resulting  from  exposure  to  hazardous 
agents.  “Ecological  risk  assessment”  usually  means  the  estimation  of  hazardous  agent  impacte  on  non¬ 
human  elements  of  the  environment  or  the  risk  associated  with  chemicals.  Very  little  has  been  written  about 
risk  and  uncertainty  analysis  applied  to  environmental  resource  planning  as  the  Corps  conducts  it. 

Other  observations  include  the  fact  that  several  federal  agencies  are  actively  involved  in  risk  analysis 
research  and  development  that  could  be  of  tremendous  Interest  to  the  Corps  in  general  and  to  the 
environmental  program  in  particular;  that  there  are  many  talented  people  doing  a  lot  of  good  work  in  risk 
analysis;  that  there  are  many  good,  if  narrowly  focused,  periodicals  available,  several  of  which  are  listed  in 
the  “Literature  Sources  to  Monitor”  section  of  Chapter  Three;  that  risk  and  uncertainty  analysis  is 
everywhere  in  the  literature,  but  it  may  be  implicit  rather  than  explicit;  and,  finally,  that  there  is  a  great  deal 
of  good  literature  from  which  to  choose.  Although  much  of  the  literature  discovered  in  the  course  of  this 
review  that  referred  to  environmental  risk  analysis  invariably  alluded  to  the  EPA  paradigm  for  assessing  risks 
to  human  health,  there  is  nonetheless  much  that  is  of  interest  to  the  Corps  in  the  literature,  a  point  made 
numerous  times  in  this  report. 
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CHAPTER  ONE:  INTRODUCTION 


INTRODUCTION 

This  literature  review  was  prepared  to  provide  assistance  to  the  U.S.  Army  Corps  of  Engineers 
(Corps)  in  beginning  to  assess  risk  and  uncertainty  in  the  evaluation  of  environmental  investments.  To 
address  this  and  other  issues  the  Corps  initiated  the  Evaluation  of  Environmental  Investments  Research 
Program  (EEIRP),  intended  to  provide  Corps  planners  with  methodologies  and  techniques  to  aid  in 
developing  supportable  environmental  restoration  and  mitigation  projects  and  plans.  The  EEIRP  has  been 
divided  into  ten  specific  study  areas  or  work  units.  One  of  these  work  units  is  “Incorporating  Risk  and 
Uncertainty  into  Environmental  Evaluation”.  The  literature  review  presented  in  this  report  was  prepared  as 
an  initial  effort  for  this  work  unit. 


TARGET  AUDIENCE 

This  literature  review  is  primarily  for  Corps  personnel,  but  it  has  two  audiences.  First,  it  serves  as 
a  primer  on  the  generic  risk  analysis  literature  that  will  help  planners  find  the  tools  they  need  to  do  risk 
analysis.  Thus,  one  audience  is  the  Corps  analyst  who  knows  little  or  nothing  about  risk  analysis,  and  who 
will  eventually  be  responsible  for  conducting  the  risk  and  uncertainty  analysis  for  environmental 
investments.  Second,  the  review  is  to  help  more  experienced  personnel  find  their  way  into  the  environmental 
resource  risk  analysis  literature.  Thus,  this  review  may  serve  as  a  gateway  to  the  more  detailed  and  specific 
risk  analysis  literature  for  the  Corps’  risks  analysts,  planners,  managers,  modelers,  and  environmental 
experts. 


ORGANIZATION  OF  REPORT 

To  serve  these  two  audiences  the  literature  review  is  organized  as  shown  in  Figure  1-1 .  Two  main 
parts  follow  this  overview.  First,  Chapter  Two  contains  a  review  of  existing  Corps  guidance  and  reports  on 
risk  analysis.  This  stands  alone  as  a  brief  guide  to  what  current  Corps  guidance  on  risk  analysis  might  have 
to  offer  analysts  applying  risk  analysis  to  environmental  investments.  To  provide  context  to  this  review,  it 
is  preceded  by  an  introduction  to  the  generic  concept  of  uncertainty  in  the  ecosystem  restoration  planning 
process. 


Next,  Chapter  Three  contains  a  review  of  the  professional  literature.  Fifty-two  books,  reports, 
papers  or  articles  have  been  reviewed  at  length  for  this  report.  They  are  presented  in  two  sections,  following 
an  introduction  that  describes  the  review  process  and  some  of  its  major  findings.  The  first  section  contains 
literature  that  addresses  the  general  methodology  of  risk  and  uncertainty  analysis.  It  introduces  readers  to 
the  literature  that  will  help  them  find  access  to  the  general  tools  and  techniques  of  risk  and  uncertainty 


1 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 
An  Annotated  Bibliography _ 


Figure  1-1:  Organization  of  Report 


analysis.  The  emphasis  here  is  on  introductory  level  readings,  though  not  to  the  exclusion  of  some 
intermediate  and  advanced  sources  of  information.  The  second  section  of  the  professional  literature  review 
includes  applications  of  risk  analysis  to  environmental  issues.  It  is  generally  a  more  demanding  literature. 

In  addition  to  the  52  summaries  provided  in  these  two  sections  there  are  two  more  sections  of  the 
professional  literature  review.  The  third  section  provides  some  suggestions  to  the  reader  for  sources  of 
information  worth  monitoring  for  future  developments  in  the  literature.  The  final  section  of  the  professional 
literature  provides  an  extended  traditional  bibliography  of  books  and  articles  of  potential  interest. 
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CHAPTER  TWO:  REVIEW  OF  EXISTING  CORPS 
RISK  ANALYSIS  GUIDANCE 


INTRODUCTION 

Since  the  inception  of  the  Corps'  Risk  Analysis  for  Water  Resources  Investments  Research  Program 
at  the  Institute  for  Water  Resources  there  has  been  an  expansion  of  the  application  of  risk  and  uncertainty 
analysis  methods  to  water  resources  planning.  Risk  and  uncertainty  analysis  has  grown  from  a  fringe 
concept  in  the  planning  process  to  a  mainstream  idea  on  the  way  to  becoming  a  powerful  tool  for  improving 
the  Corps'  decision-making  process.  During  the  course  of  this  evolutionary  process  the  Corps  has  begun 
to  develop  guidance  on  risk  analysis  evaluation  frameworks  for  flood  control,  lock  and  dam  rehabilitation 
and  other  areas  related  to  the  Corps'  missions. 

This  research  represents  an  initial  step  in  the  development  of  a  risk  analysis  framework  for 
environmental  investments.  The  first  task  in  this  research  was  to  review  the  Corps'  existing  risk  analysis 
guidance  to  determine  how  it  may  be  useful  in  crafting  the  future  development  of  risk  and  uncertainty 
analysis  guidelines  for  evaluating  environmental  investments. 

The  review  begins  with  an  introduction  to  the  concept  of  ecosystem  restoration.  To  provide  a 
contextual  approach  to  the  review,  the  generic  notion  of  uncertainty  is  considered  in  the  ecosystem 
restoration  planning  process.  A  taxonomy  of  uncertainty  is  offered  as  a  starting  point  for  understanding  the 
types  and  sources  of  uncertainty  encountered  in  the  environmental  resources  planning  process.*  This  is 
provided  to  aid  the  reader  approaching  the  notion  of  uncertainty  for  the  first  time  with  seriousness  of 
purpose.^ 

The  context  begins  with  a  review  of  two  different  ways  to  look  at  probabilities.  Next,  there  is  a 
generic  discussion  of  the  types  of  quantities  encountered  in  ecosystem  restoration  planning  that  can  be 
uncertain.  The  different  types  of  quantities  that  are  uncertain  can  have  different  sources  of  uncertainty,  so 
a  discussion  of  the  sources  of  uncertainty  follows.  The  contextual  information  looks  at  possible  sources  of 
uncertainty  that  may  be  encountered  in  an  ecosystem  restoration  planning  framework.  The  remainder  of 
Chapter  Two  comprises  the  review  of  specific  Corps'  risk  analysis  guidance.  Items  summarized  in  a 


'  Throughout  this  review  the  terms  environmental  resources  planning,  environmental  investment  planning, 
environmental  restoration  planning,  and  ecosystem  restoration  planning  are  used  interchangeably. 

2 

For  a  more  detailed  investigation  of  the  sources  of  uncertainty  in  environmental  restoration  planning,  as  well 
as  some  of  the  tools  available  to  address  these  uncertainties,  please  refer  to  another  EEIRP  product.  An  Introduction  to 
Risk  and  Uncertainty  in  the  Evaluation  of  Environmental  Investments,  IWR  Report  96-R-8,  March  1996. 
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literature  review  in  Chapter  Three  of  this  report  are  occasionally  referred  to  in  this  review  of  existing  Corps’ 
risk  analysis  guidance.  The  last  name  of  the  first  author  listed  and  year  of  publication  is  cited  in 
parenthetical  notes  following  the  mention  of  selected  topics. 

ECOSYSTEM  PLANNING  IN  THE  CIVIL  WORKS  PROGRAM 

The  starting  point  for  discussing  the  development  of  risk  and  uncertainty  analysis  for  environmental 
investments  is  the  Corps’  current  framework  for  such  planning.  Engineer  Circular  1 105-2-206,  "Ecosystem 
Restoration  Planning  in  the  Civil  AVorks  Program",  provides  the  description  of  the  planning  framework 
against  which  this  review  of  risk  analysis  guidance  proceeds. 

The  purposes  of  EC  1 105-2-206  are  to  assure  that  Civil  Works  investments  in  ecosystem  restoration 
will  have  the  intended  beneficial  effects;  are  consistent  with  Administration  policy;  and  will  be  conducted 
in  the  most  cost  effective  manner.  Breaking  with  the  environmental  mitigation  tradition  of  mitigating 
damage  to  fish  and  wildlife  only,  studies  and  projects  initiated  under  the  Corps'  program  to  restore  ecological 
resources  will  be  formulated  and  implemented  under  principles  of  ecosystem  restoration  (see  Cairns  1993, 
Droppo  1993,  and  Kusler  1990  in  Chapter  Three).  Ecosystem  restoration  provides  a  more  comprehensive 
approach  for  assessing  and  addressing  the  problems  associated  with  disturbed  and  degraded  ecological 
resources.  The  primary  goal  of  ecosystem  restoration  is  to  return  an  ecosystem’s  structure,  function,  and 
dynamic  processes  to  a  less  degraded,  natural  condition. 

Ecosystem  restoration  planning  considers  the  roles  of  plant  and  animal  species  and  their  habitats  in 
larger  community  and  ecosystem  frameworks.  Projects  developed  in  this  framework  are  conceived  in  a 
comprehensive  context  that  provides  aquatic,  wetland  and  upland  complexes  with  the  potential  for  long-term 
survival  as  functioning  systems. 

Overlaying  the  principles  of  ecosystem  restoration  on  the  Corps'  existing  plan  formulation  and 
reporting  processes  results  in  a  planning  environment  that  is  fraught  with  uncertainty.  In  the  next  section 
we  discuss  some  of  the  potential  sources  of  uncertainty  in  the  ecosystem  planning  process. 

UNCERTAINTY  AND  ECOSYSTEM  RESTORATION  PLANNING 

To  discuss  uncertainty  in  the  context  of  ecosystem  restoration  planning  we  have  to  address  two 
significant  issues.  First,  there  is  the  taxonomy  of  risk  and  uncertainty  analysis.  What  kinds  of  uncertainty 
are  there?  We  need  to  know  something  about  uncertainty  in  order  to  know  where  and  how  to  look  for  it. 
Second,  we  need  to  know  where  in  the  Corps  planning  process  it  might  exist.  Only  then  can  we  turn  toward 
the  Corps'  existing  guidance  to  see  if  it  can  be  helpful  in  the  development  of  risk  and  uncertainty  analysis 
guidelines  for  environmental  investments. 

This  section  begins  with  a  discussion  of  the  taxonomy  of  uncertainty  that  will  be  used  in  this  report. 
It  is  the  taxonomy  used  by  Morgan  and  Henrion  in  their  benchmark  1992  book.  Uncertainty:  A  Guide  to 
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Dealing  With  Uncertainty  in  Quantitative  Risk  and  Policy  Analysis  (reviewed  in  Chapter  Three  of  this  report 
and  on  which  much  of  this  section  is  based).  Next,  we  offer  a  way  to  look  at  the  places  in  the  Corps' 
ecological  restoration  planning  process  where  significant  uncertainties  of  the  types  identified  in  the 
taxonomy  might  occur. 

RISK  AND  UNCERTAINTY 

Uncertaintydescribes  any  situation  in  which  we  are  not  absolutely  sure.  Risk  describes  a  situation 
in  which  there  is  a  chance  of  something  bad  happening.  The  chance  is  often  unknown,  as  are  the  nature  and 
extent  of  the  “bad”  thing  that  might  happen.  Risky  situations,  are,  then,  characterized  by  uncertain  likelihood 
of  occurrence  and  uncertain  consequences;  clearly,  a  case  of  uncertainty.^  For  simplicity  this  section 
proceeds  by  considering  uncertainty  to  be  inclusive  of  risk  in  a  rather  broad  sense  (see  Glockman  1993). 

Uncertainty  arises  for  a  variety  of  reasons:  incomplete  information,  disagreement  among  experts 
or  data  sources,  imprecise  language,  or  variability  that  results  from  sampling  error  or  the  structure  of  a 
model.  Probability  is  often  used  as  a  measure  of  uncertain  belief  Probability  is  an  appropriate  way  to 
express  some  kinds  of  uncertainty  (for  example,  the  uncertainty  concerning  the  occurrence  of  a  flood),  but 
not  others  (e.g.,  uncertainty  regarding  an  appropriate  mitigation  goal)  (see  Morgan  1992). 

THE  NATURE  OF  PROBABILITY 

Probability  is  the  best  known  way  to  quantify  uncertainty.  There  are  two  basic  views  of  probability: 
the  classical  or  frequentist  view  and  the  subjectivist  or  Bayesian  view  (see  Baird  1989).  In  ecosystem 
restoration  planning  there  will  be  occasions  to  use  both  views,  so  they  are  discussed  briefly  below. 

Classical  View  of  Probability 

The  classical  or  frequentist  view  of  probability  (see  Megill  1984)  defines  the  probability  of  an  event 
as  the  frequency  with  which  it  occurs  over  a  very  long  series  of  similar  trials.  Probability  can  be  estimated 
by  the  frequency  of  an  event  within  a  sample  of  that  very  long  series.  For  example,  the  probability  of  a  flow 
of  25,000  cubic  feet  per  second  (cfs)  or  greater  is  the  frequency  with  which  it  is  observed  to  occur  over  a 
very  long  period  of  time.  If  we  have  a  sample  of  1 00  years  of  flow  data  and  see  that  flows  equal  to  or  greater 
than  25,000  cfs  occurred  10  times,  the  frequency  of  this  flow  is  10/100  or  0.1,  a  ten-year  event. 


^  The  authors  acknowledge  that  these  simple  definitions  mask  distinctions  between  the  terms  that  may  be 
employed  elsewhere.  For  example,  “risk  situations”  have  been  described  as  those  in  which  the  potential  adverse  outcomes 
can  be  described  in  reasonably  well-known  probability  distributions,  while  “uncertain  situations”  cannot  be  described  by 
objectively  known  probability  distributions.  The  language  of  risk  and  uncertainty  analysis  can  be  very  confusing  because 
many  disciplines  make  use  of  its  tools  and  methods.  The  point  to  be  made  here  is  that  how  we  label  a  situation  in  the 
absence  of  certainty  is  less  important  than  that  we  address  it.  The  terms  risk  analysis,  risk  and  uncertainty  analysis,  and 
uncertainty  analysis  are  used  as  virtual  synonyms  in  this  report,  less  because  there  is  no  difference  than  because  the 
difference  is  relatively  unimportant. 
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But  the  frequency  of  flows  in  this  sample  are  only  an  estimate  of  the  probability  of  flows  over  25,000 
cfs.  Probability  is  the  value  to  which  the  long-run  frequency  converges  as  the  number  of  trials  increases. 
Suppose  the  true  probability  of  a  flow  of  25,000  cfs  or  more  is  0.2.  By  pure  chance,  our  100  years  of  record 
do  not  reflect  the  true  probability.  As  the  period  of  record  (i.e.,  the  number  of  trials,  in  this  case,  years  of 
record)  gets  longer  and  longer,  the  observed  frequency  of  these  flows  will  begin  to  approach  the  true 
probability. 

In  this  view,  probability  is  seen  as  an  intrinsic  property  of  the  system  that  generates  the  events. 
Flow  volume  is  a  function  of  rainfall,  groundwater  contributions,  and  channel  and  basin  characteristics.  Over 
a  very  long  period  of  time  flows  greater  than  or  equal  to  25,000  cfs  can  be  expected  in  20  percent  of  the  years 
because  of  the  combination  of  local  precipitation  probabilities  and  the  fixed  characteristics  of  the  basin. 
Frequentists  might  observe  flows  over  a  period  of  time  and  from  that  data  try  to  estimate  the  true  probability 
with  more  or  less  success,  depending  on  the  period  of  time  they  monitor,  the  length  of  that  time  period,  and 
it's  chance  representativeness  of  the  true  probability. 

The  classical  and  frequentist  views  can  be  distinguished  by  their  attitudes  toward  the  probability 
estimate.  The  classical  view  sees  probability  as  an  intrinsic  property  of  a  system.  For  example,  if  we  breed 
two  heterozygous  plants  there  is  a  0.25  probability  that  an  offspring  will  exhibit  the  recessive  characteristic. 
This  is  a  probability  in  a  classical  sense.  The  value  0.25  is  an  intrinsic  property  of  the  system  of  two 
heterozygous  plants."  The  frequentist  view  is  demonstrated  in  the  stream  flow  example  above  where  we  have 
flow  data  from  a  period  of  record  and  observe  how  often  the  event  of  interest  has  occurred.  The  resulting 
frequency  is  based  on  what  we  have  observed  and  not  necessarily  on  what  is  intrinsic  to  the  system  we  are 
observing. 

The  frequentist  view  of  probability  will  arise  often  in  ecosystem  restoration.  The  pure  classical  view 
of  probability  will  be  observed  less  often. 

Subjective  View  of  Probability 

The  subjective/  personalist/  Bayesian  view  of  probability  is  quite  different  from  the  classical/ 
frequentist  view.  The  subjective  view  of  the  probability  that  an  event  will  occur  is  the  degree  of  belief  that 
a  person  has  that  it  will  occur,  given  all  the  information  currently  available  to  and  known  by  that  person. 
Probability  depends  not  only  on  the  event  itself  but  also  on  the  state  of  information  about  that  event. 

In  symbols  we  would  write  a  classical  probability  for  event  X  occurring  as  P(X).  The  subjective 
probability  would  be  P(X|e)  where  X  is  the  event  and  e  is  the  person's  state  of  information  upon  which  the 
strength  of  his  or  her  belief  is  conditioned. 


"  A  heterozygous  plant  has  one  dominant  and  one  recessive  gene,  Rr.  When  two  such  plants  are  bred  the 
offspring  can  be  RR,  Rr,  Rr,  or  rr.  There  is  one  chance  in  four  of  observing  the  recessive  characteristic,  rr. 
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Implicit  in  this  view  of  probability  are  at  least  three  important  points.  First,  since  different  people 
may  have  different  states  of  information  and/or  different  strengths  of  belief  that  something  will  or  will  not 
happen,  it  is  quite  normal  that  different  people  would  estimate  different  probabilities  for  the  same  event. 
Second,  because  the  same  person  can  acquire  new  information  over  time  the  same  person  may  change  his 
or  her  estimate  of  the  probability  of  an  event  occurring.  Third,  following  from  these  first  two  points  we 
conclude  that  there  is  no  such  thing  as  "the"  probability  of  an  event.  This  is  very  different  from  the  classical 
view  that  holds  that  probability  is  an  intrinsic  property  of  the  system  from  which  the  event  of  interest  is 
generated. 

In  ecosystem  restoration  planning,  many  of  the  probabilities  will  be  subjective  in  nature.  Given  the 
substantial  scientific  uncertainty  attending  this  emerging  field  it  will  not  be  uncommon  to  see  probabilities 
expressed  as  an  expert's  degree  of  belief  that  an  event  will  or  will  not  occur  (see  Baird  1989,  Clemen  1990, 
Megill  1984,  and  Morgan  1992).  The  very  notion  of  an  adaptive  management  approach  to  planning,  the 
approach  recommended  in  EC  1 105-2-206,  is  conceptually  aligned  with  the  subjectivist  view  of  probability. 
As  additional  information  is  obtained  probabilities  are  revised  in  the  subjectivist  view.  As  additional 
information  is  obtained  management  techniques  are  refined  in  the  adaptive  management  approach.  Hence, 
there  should  be  little  philosophical  difficulty  with  the  notion  of  subjectivist  probabilities. 

The  Union  of  the  Two  Views 

In  theory,  when  there  is  sufficient  data  for  the  frequentist  to  estimate  a  probability,  the  subjectivist's 
assessment  of  his  or  her  probability  will  converge  to  the  frequentist's  estimate  of  "the"  probability  of  the 
event.  In  other  words,  they  will  tend  to  agree  as  information  becomes  available. 

UNCERTAINTY  AND  GENERIC  QUANTITIES  IN  ECOSYSTEM  RESTORATION  PLANNING 

It  has  been  suggested  that  probability  can  be  used  to  express  uncertainty  and  that  subjective 
probabilities  can  be  helpful  in  expressing  uncertainty  about  quantities.  Planning  models  contain  many 
different  types  of  quantities  and  not  all  of  them  are  amenable  to  probabilistic  analysis.  In  general,  only 
empirical  quantities  should  be  represented  in  probabilistic  terms.  To  see  why  this  may  be  so  we  need  to 
consider  some  of  the  types  of  quantities  that  may  be  encountered  in  ecosystem  restoration  planning  models 
(see  Morgan  1992).  Table  2-1  presents  a  summary  of  the  generic  types  of  quantities  that  can  be  encountered 
in  the  planning  framework. 

Empirical  parameters  or  chance  variables  are  empirical  quantities  that  represent  measurable 
properties  of  the  resources  and  factors  that  can  be  measured  in  the  planning  process.  They  include  quantities 
in  the  domains  of  all  the  disciplines  involved  in  the  study,  such  as  population  counts  for  significant  species, 
water  temperatures,  stream  flows,  and  quantity  and  cost  estimates,  for  example. 

In  principle,  all  empirical  quantities  should  have  a  single,  true  value  that  is  empirically  measurable. 
For  example,  the  white  tail  deer  population  could,  in  theory,  be  measured  exactly  by  counting.  There  is  a 
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TABLE  2-1 :  Generic  Types  of  Quantities  in  the  Planning  Framework 

Type  of  quantity 

Example 

Treatment  of  uncertainty 

Empirical  parameter  or  chance 
variable 

Mortality  rate,  stream  discharge 

Probabilistic,  parametric^ 

Defined  constant 

Gallons  per  acre-foot 

Certain  by  definition 

Decision  variable 

Level  of  protection,  mitigation  goal 

Parametric 

Value  parameter 

Discount  rate,  value  of  life 

Parametric 

Index  variable 

Acreage,  longitude  &  latitude 

Certain  by  definition 

Model  domain  parameter 

Study  area,  planning  horizon,  base  year 

Parametric 

Outcome  criterion 

Benefit/  Cost  ratio,  incremental  costs 

Depends  on  treatment  of 
inouts 

single  true  value  for  the  white  tail  deer  population  at  a  given  place  and  point  in  time.  Whether  we  can 
actually  determine  that  true  value  is  another  matter. 

One  type  of  quantity  is  always  certain:  defined  constants.  There  will  always  be  43,560  square  feet 
to  an  acre,  326,000  gallons  of  water  in  an  acre-foot.  These  values  are  true  by  definition  and  we  never  have 
to  worry  about  their  uncertainty. 

Decision  variables  are  quantifiable  achievement  levels;  they  are  set  by  the  planning  team.  They  are 
sometimes  called  “control  variables”  or  “policy  variables”.  We  may  be  uncertain  about  whether  we  have 
chosen  the  best  level  of  protection  for  a  flood  damage  reduction  project  or  the  best  level  of  environmental 
mitigation.  It  makes  no  sense,  however,  to  be  uncertain  about  the  "true"  value  of  level  of  protection  or 
environmental  mitigation  because  these  quantities  have  no  true  values.  These  are  decision  variables.  Their 
values  are  selected  by  the  planning  team.  To  represent  them  as  probabilistic  variables  detaches  them  from 
the  decision  process.  It  is  generally  more  appropriate  to  vary  decision  variables  parametrically,  i.e.,  to  use 
a  sensitivity  analysis. 


^  Parametric  variation,  also  known  as  sensitivity  analysis,  is  a  process  in  which  a  number  or  parameter  that  is 
treated  like  a  constant  in  the  analysis  is  varied  systematically  over  a  range  of  values  to  examine  its  impact  on  the  decision 
or  result. 
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It  is  important  to  bear  in  mind  that  decisions  are  made  throughout  the  planning  process.  Planners 
may  be  required  to  make  a  choice  and  select  one  course  of  action  over  another  at  various  points  in  the 
planning  process.  It  may  be  desirable  to  test  the  sensitivity  of  the  results  of  the  planning  process  to  the 
alternative  courses  of  action  in  more  cases  than  it  is  possible  to  do  so. 

Value  parameters  represent  the  preferences  of  the  planning  team  or  the  people  they  represent,  e.g., 
the  general  public,  resource  agencies,  higher  authority.  The  discount  rate  is  an  example  of  a  value  parameter. 
Historically,  there  has  been  considerable  debate  about  the  proper  discount  rate.  In  recent  ecological  literature 
the  debate  has  been  revisited.  Congress  has  made  a  value  judgment  about  what  the  discount  rate  should  be 
and  has  prescribed  a  methodology  for  calculating  it. 

It  is  not  unusual  for  a  value  parameter  to  be  treated  as  an  empirical  quantity  and  represented  by  a 
probability  distribution.  This,  however,  is  usually  a  mistake,  sometimes  a  serious  one.  The  key  point  about 
value  parameters  is  that  they  are  value  judgments.  If  the  decision-maker  is  not  sure  what  he  prefers,  it  would 
rarely  be  helpful  to  leave  the  value  judgment  to  chance,  as  would  be  done  if  it's  treated  like  an  empirical 
quantity.  If  the  planning  team  is  not  sure  about  a  value  parameter  it  is  better  to  vary  the  values 
parametrically,  i.e,  first  try  one,  then  another,  and  so  on.  This  allows  the  team  to  learn  how  changes  in  the 
parameter's  value  affect  the  outcomes  in  which  we  are  interested. 

Value  parameters  often  tend  to  be  those  people  are  least  sure  about  and  they  may  contribute 
significantly  to  the  uncertainty  about  the  best  alternative.  If  these  quantities  are  treated  as  probabilistic  it 
may  mask  the  effect  of  the  different  value  choices  on  the  planning  objectives.  A  systematic  sensitivity 
analysis  can  be  far  more  illuminating  and  can  help  decision-makers  better  understand  and  refine  their 
preferences. 

Index  variables  are  used  to  identify  and  locate  information  in  a  model.  For  example,  the  base  year, 
project  year  10,  grid  cell  230,  90o50'  longitude  and  30o42'  latitude.  There  is  no  reason  to  be  uncertain  about 
the  value  of  an  index  variable.  Although  we  may  have  considerable  uncertainty  about  matters  of  space  and 
time  -  when  and  where  something  might  happen  -  it  makes  no  sense  whatsoever  to  design  an  indexing  system 
that  we  are  uncertain  about. 

Model  domain  parameters  specify  the  scope  of  the  study.  We  identify  the  Potomac  watershed  as 
the  study  area  and  a  50-year  planning  horizon.  Values  like  these  define  the  extent  of  the  study.  The  planning 
horizon  delimits  the  temporal  domain  of  the  study,  the  study  area  delimits  the  spatial  domain.  Considering 
levee  heights  in  two-foot  increments  and  habitat  units  in  increments  of  100  are  examples  of  other  model 
domain  parameters. 

It  would  be  quite  normal  to  be  uncertain  about  what  domain  values  are  appropriate.  As  with  the 
value  parameters,  however,  there  are  no  true  values  and  it  would  be  inappropriate  to  represent  these  model 
domain  parameters  with  probability  distributions.  The  choice  of  the  domain  values  is  up  to  the  planning 
team.  In  many  cases  these  decisions  have  been  made  and  are  set  forth  in  the  various  sources  of  planning 
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guidance.  Nonetheless,  it  can  often  be  useful  to  vary  these  values  parametrically  to  determine  the  sensitivity 
of  study  results  to  different  model  domains. 

The  use  of  parametric  sensitivity  analysis  (see  Clemen  1990,  Morgan  1992)  in  ecosystem  restoration 
planning  would  appear  to  be  a  most  valuable  tool.  In  the  first  blush  of  excitement  with  risk  and  uncertainty 
analysis,  many  Corps  planners  have  leapt  to  embrace  Monte  Carlo  simulation  models*  (see  Allardyce  1985, 
Clemen  1990,  Davis  1982,  Morgan  1992,  and  Newendorp  1976).  While  these  models  are  undeniably  useful 
in  many  settings,  they  are  not  appropriate  in  all  settings. 

Planners  involved  in  ecosystem  restoration  have  espoused  an  adaptive  management  approach.  The 
success  of  this  approach  is  predicated  on  planners  becoming  more  informed  about  ecosystem  restoration  as 
the  results  of  completed  projects  are  monitored.  Risk  and  uncertainty  analysis  can  help  inform  planners 
about  ecosystem  restoration.  The  techniques  can  reveal  insights  as  they  improve  decisions.  It  is  usually 
easier  to  see  the  implications  of  value  and  domain  choices,  for  example,  by  parametrically  varying  these 
values  in  a  sensitivity  analysis.  Only  in  this  way  can  we  learn  such  basic  things  we  need  to  know  as  what 
planning  horizons  make  sense  and  what  discount  rates  are  most  appropriate. 

The  outcome  criteria  are  the  variables  we  use  to  rank  or  measure  the  desirability  of  possible 
outcomes.  The  Corps  has  traditionally  used  the  benefit-cost  ratio  (BCR)  in  its  planning  studies.  When 
possible,  the  BCR  is  equally  useful  for  environmental  investment  planning.  Incremental  costs  are  another 
outcome  criteria  in  common  usage  for  environmental  resources  planning.  These  outcome  criteria  will  be 
deterministic  if  there  is  no  uncertainty  in  the  inputs  used  to  develop  them  and  there  is  an  unambiguous 
analytical  method  to  develop  the  criteria.  However,  if  the  inputs  are  probabilistic  or  there  are  no  analytical 
methods  for  estimating  the  criteria,  the  outcome  criteria  will  be  probabilistic  (see  Bishop  1978  and  Templet 
1994). 

Table  2-2  presents  some  generic  environmental  investment  study  tasks  and  concepts  and  attempts 
to  indicate  the  nature  of  the  quantities  that  might  be  encountered  in  these  tasks.  Many  have  more  than  one 
entry  because  the  items  in  most  cases  could  be  broken  down  into  numerous  specific  tasks,  each  of  which 
could  involve  a  different  type  of  quantity.  A  table  of  this  type  could  be  a  useful  device  for  helping  analysts 
think  about  the  types  of  quantities  they  are  working  with  and  the  subsequent  methods  for  addressing  their 
uncertainty. 


*  The  Monte  Carlo  process  is  frequently  used  in  computer  simulation  models  in  which  values  for  random 
variables  are  generated  based  on  one  or  more  specific  probability  distributions  (e.g.,  normal,  lognormal,  or  uniform 
distributions).  The  process  works  as  if  the  user  specifies  a  type  of  probability  distribution  and  its  parameters  and  a 
number  is  randomly  selected  from  that  distribution.  The  random  numbers  generated  by  a  Monte  Carlo  process  can 
represent  a  sample  of  potential  results  in  the  process  under  consideration. 
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TABLE  2-2:  Environmental  Investment  Study  Tasks/Concepts 

Types  of  Quantities 

Study  Task/Concept 

Empirical 

Constant 

Decision 

Value 

Index 

Domain 

Outcome 

Investment  costs 

X 

“ 

- 

- 

- 

- 

- 

Beneficial  impacts 

X 

- 

- 

- 

- 

- 

X 

Watershed 

- 

- 

- 

- 

- 

X 

- 

Indicator  species 

- 

- 

X 

- 

- 

- 

- 

Antecedent  ecological  condition 

X 

X 

X 

X 

X 

- 

Role  of  species  in  ecosystem 

X 

- 

X 

- 

- 

- 

- 

Resource  significance 

X 

- 

X 

X 

- 

- 

- 

Restoration  objectives 

- 

- 

X 

- 

- 

- 

Inventory 

X 

X 

- 

X 

X 

X 

- 

Forecast 

X 

- 

- 

- 

- 

- 

Influencing  factors 

X 

- 

X 

X 

- 

- 

- 

Ecological  interactions 

X 

- 

- 

- 

X 

X 

- 

NED  impacts 

X 

- 

X 

X 

X 

X 

Experimental  technology 

X 

- 

X 

X 

- 

- 

- 

Probability  of  success 

X 

- 

- 

- 

- 

- 

- 

Consequence  of  failure 

X 

- 

- 

- 

- 

- 

- 

SOURCES  OF  UNCERTAINTY 


To  this  point  we  have  concentrated  primarily  on  the  uncertainty  that  resides  in  empirical  quantities. 
That  is  perhaps  the  most  familiar  kind  of  uncertainty  but  it  may  not  be  the  most  important.  Model  uncertainty 
is  another  type  of  uncertainty  that  can  be  more  important  in  many  cases  (see  Morgan  1992). 

Using  the  habitat  evaluation  procedures  (HEP),  we  can  envision  many  kinds  of  empirical  uncertainty. 
There  will  be  considerable  uncertainty  about  the  choice  of  indicator  species,  the  data,  and  so  forth.  It  is  not 
difficult  to  envision  uncertainty  about  the  values  of  the  quantities  that  have  to  be  entered  into  the  HEP  model. 
It  is  much  more  difficult  to  try  to  think  about  the  uncertainty  in  the  model  itself.  How  much  uncertainty  is 
there  in  the  form  of  the  model?  Are  the  assumptions  underlying  the  model  reasonable?  Unassailable?  How 
much  uncertainty  is  there  in  the  validity  of  identifying  an  indicator  species  to  represent  the  functioning  of 
an  ecosystem?  How  much  uncertainty  is  there  in  tying  the  future  viability  of  a  species  to  a  habitat  suitability 
index?  Are  these  even  proper  ways  to  think  about  the  problems? 
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MODEL  FORM  UNCERTAINTY 

Uncertainty  about  model  form  frequently  reflects  a  lack  of  theory  or  data,  or  perhaps  a  disagreement 
among  experts  about  the  underlying  scientific  or  technical  mechanisms  at  work  in  the  system  we  are  trying 
to  model.  Given  the  evolving  art  of  ecosystem  restoration  planning  there  would  appear  to  be  substantial 
model  uncertainty  present  in  many  environmental  investment  planning  activities.  The  biological  and  habitat 
models  currently  in  use  can  be  a  significant  source  of  uncertainty  in  the  evaluation  of  environmental 
resources  investment  plans,  and  will  most  likely  be  investigated  as  environmental  risk  assessment  research 
proceeds 

UNCERTAINTY  ABOUT  EMPIRICAL  QUANTITIES 

Empirical  quantities  measure  properties  of  the  real  world  systems  analyzed  or  modeled  in  an 
environmental  investment  study.  A  starting  point  for  most  risk  and  uncertainty  analysis  would  be  to  identify 
those  key  variables  and  parameters  under  each  of  the  quantities  listed  in  Table  2-1  (see  Morgan  1992  and 
Canter  1993).  Though  a  useful  starting  point,  merely  identifying  the  key  variables  does  not  distinguish  the 
causes  of  uncertainty  in  the  key  variable.  Analysts  need  a  way  to  think  about  the  causes  of  uncertainty. 

Generic  sources  of  uncertainty  in  empirical 
quantities  are  shown  in  Table  2-3.  These  can  be 
used  to  order  one's  thinking  about  how  to  approach 
the  risk-based  analysis.  By  systematically 
identifying  the  sources  of  uncertainty  in  a  study  and 
devising  an  analysis  to  address  the  most  significant 
sources  of  uncertainty,  a  good  analysis  can  be 
assured. 

Statistical  Variation  and  Random  Error 

The  best  understood  source  of  uncertainty 
is  the  random  error  or  statistical  variation  that 
arises  from  errors  in  the  direct  measurement  of  a 
quantity.  Few  measurements  are  exactly  right.  Our 
instruments  are  not  properly  calibrated,  our  powers 
of  observation  are  imperfect,  we  are  flawed  human 
beings.  Our  tools  and  ourselves  make  mistakes. 

This  kind  of  uncertainty  is  commonly  expressed 
through  standard  deviations  and  confidence 
intervals.  Examples  of  this  type  of  uncertainty  in 
environmental  investment  will  include  estimates  of  ground  cover  and  infiltration  of  water  through  soil  t)q)es. 


TABLE  2-3:  Generic  Sources  of  Uncertainty 
in  Empirical  Quantities 

•  Random  error  and  statistical  variation 

•  Systematic  error  and  subjective  judgment 

•  Linguistic  imprecision 

•  Variability 

•  Randomness  and  unpredictability 

•  Disagreement 

•  Approximation 

Source:  Morgan  &  Henrion  1992 
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Subjective  Judgement  and  Systematic  Error 

Subjective  judgements  are  common  sources  of  systematic  errors.  So-called  experts  are  frequently 
ignorant  of  their  own  biases  in  estimating  or  assessing  quantitative  measures.  Experts  can  sometimes  be  the 
last  to  recognize  the  limits  of  their  experience  and  knowledge. 

Systematic  errors  arise  due  to  biases  in  the  measurement  instrument  or  the  individual  taking  the 
measurement.  Attempts  to  estimate  populations  from  animal  excrement  counts  can  lead  to  errors  if  the  ratio 
of  excrement  to  organisms  is  incorrect.  Estimating  values  with  inappropriate  or  poorly  calibrated  tools  yields 
systematic  errors.  Systematic  error  or  bias  is  defined  as  the  difference  between  the  true  value  of  the  quantity 
being  measured  (the  actual  number  of  animals)  and  the  value  to  which  the  mean  of  the  measurements 
converge  as  more  observations  are  collected  (the  estimated  population  based  on  our  counting  method). 

A  good  analyst  is  careful  to  identify  and  eliminate  all  sources  of  systematic  error.  If  systematic 
error  is  recognized  it  can  generally  be  eliminated.  When  systematic  error  is  not  recognized  it  can  present 
a  serious  problem.  Estimating  the  size  of  a  systematic  error  that  is  merely  suspected  is  a  difficult  and  often 
subjective  task.  It  is  much  easier  to  underestimate  the  existence  or  effect  of  systematic  errors  than  to 
overestimate  them  when  quantifying  uncertainties.  Unfortunately,  systematic  error  is  not  reduced  by  more 
observations. 

Systematic  errors  can  occur  in  any  values  that  have  to  be  physically  measured  with  equipment  that 
needs  periodic  calibration  (e.g.,  hand-levels,  the  length  of  a  person's  pace,  flow  meters)  or  that  are  estimated 
by  people  unaware  of  their  imprecision  (e.g.,  habitat  suitability  indices).  The  solutions  to  these  types  of 
uncertainty  are  well-calibrated  equipment  in  sufficient  quantity  for  all  analysts  and  rigorous  training  in  data 
collection  techniques.  New  analysts,  contractors  with  minimally  trained  field  personnel,  and  inflexible 
experts  are  common  sources  of  such  errors. 

Linguistic  Imprecision 

Language  is  frequently  an  imprecise  vehicle  for  communication.  The  stated  objective  of  an 
environmental  project  of  “restoring  aquatic  plant  beds  for  fish  and  wildlife  habitat”  may  sound  like  a 
perfectly  clear  planning  objective  to  some  planners,  but  what  aquatic  plants  does  it  refer  to?  What  fish  and 
wildlife  habitat  would  benefit  from  the  restoration  of  aquatic  plant  beds?  The  natural  environment  would 
be  restored  to  what  level? 

In  many  cases,  a  bit  of  clear  thinking  and  precise  language  can  clear  up  most  uncertainty  stemming 
from  imprecise  communication.  Unfortunately,  when  a  new  jargon  is  developing  as  it  is  with  the 
environmental  investments  and  ecosystem  restoration  program  there  is  going  to  be  considerable  imprecision 
in  the  language  and  considerable  uncertainty  about  its  meaning.  There  is  a  great  deal  of  variation  in  the  way 
different  people  interpret  words  and  phrases  and  their  interpretation  is  very  context-dependent. 
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Even  words  in  common  usage  like  ecosystems,  resources,  and  functioning  have  clouded  meanings  in 
new  contexts.  What  is  a  functioning  ecosystem?  Would  lumbermen  give  the  same  answers  as  an  Audubon 
Society  spokesperson  when  describing  the  functions  of  the  forests  of  the  Pacific  Northwest?  That  linguistic 
imprecision  that  permits  two  people  to  give  widely  varying  and  honest  definitions  to  the  same  terminology  is 
a  significant  source  of  uncertainty. 

Fortunately,  linguistic  imprecision  can  be  relatively  easily  avoided  once  its  potential  existence  is 
recognized.  It  is  imperative  that  environmental  planners  recognize  that  there  is  considerable  potential  for  such 
uncertainty  in  the  early  years  of  environmental  planning  as  a  new  formal  planning  activity.  Pains  should  be 
taken  to  assure  that  all  stakeholders  have  a  clear,  specific  and  similar  understanding  of  all  important  terms, 
concepts,  tools,  and  quantities  encountered  in  the  planning  process. 

Variability 

Variability  has  been  called  sampling  error  in  the  traditional  textbooks.  Analysts  need  to  know  the 
difference  between  uncertainty  and  variability.  Uncertainty,  in  this  current  context,  results  from  incomplete 
scientific  or  technical  knowledge;  variability  results  from  sampling  from  a  population. 

Suppose  we  want  to  model  the  uncertainty  about  the  average  size  of  striped  bass.  We  could  catch, 
weigh,  measure,  tag,  and  release  every  bass  in  the  study  area.  Or,  we  could  take  a  sample  of  striped  bass, 
weigh  and  measure  them,  and  make  some  inferences  about  the  population.  Our  uncertainty  with  the  sample 
has  two  components.  One  is  the  uncertainty  about  the  distribution  of  the  population  (uncertainty  that  arises 
from  incomplete  knowledge)  and  the  other  is  the  uncertainty  that  results  from  sampling  from  this  distribution 
(variability).’ 

When  we  take  a  sample  and  calculate  the  sample  mean  it’s  reasonable  to  expect  the  sample  mean  to 
differ  from  the  population  mean  by  some  amount.  When  a  sample  is  drawn,  sample  statistics  estimated,  and 
confidence  intervals  constructed,  the  analyst  is  treating  the  variability  in  the  data.  This  is  only  one  source  of 
uncertainty  in  the  data.  Although  it  was  common  to  address  only  this  type  of  uncertainty  in  early  risk  analyses, 
it  is  rarely  adequate.  The  other  sources  of  uncertainty  are  often  far  more  significant. 

Generally,  variability  can  be  reduced  by  disaggregating  or  increasing  the  sample  size.  Stratifying  the 
sample  or  sampling  more  units  may  help.  On  the  other  hand,  uncertainty  can  only  be  reduced  by  further 
research  that  increases  our  knowledge  of  the  uncertainty  modeled. 

Inherent  Randomness  and  Unpredictability 

Inherent  randomness  is  often  defined  as  uncertainty  that  is  impossible  to  reduce.  Some  would  debate 
this  point,  saying  that  such  indeterminacy  is  not  a  matter  of  principle  but  only  the  result  of  our  limited 


’  This  uncertainty  may  be  expressed  in  terms  of  probability  distributions  for  the  frequency  distribution  s 
parameters  such  as  its  mean,  standard  deviation,  and  median. 
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understanding  of  the  world.  The  practical  issue  for  the  analyst  is,  “Is  the  uncertainty  reducible  in  practice?” 
A  quantity  that  is  random  to  one  person  who  does  not  understand  the  systems  at  work  may  be  deterministic 
for  another  person  who  knows  the  underlying  process  that  generated  the  quantity.  The  planner  should  try 
to  distinguish  uncertainty  that  can  be  reduced  by  further  study  from  that  which  is  inherently  random,  for  the 
latter  can't  be  reduced.  Taking  full  advantage  of  the  expertise  present  in  the  interdisciplinary  study  team  can 
help  reduce  this  type  of  uncertainty  to  the  maximum  practical  extent. 

An  economist  may  not  understand  the  computed  water  surface  profiles,  and  conclude  the  uncertainty 
here  cannot  be  reduced  further.  A  hydraulic  engineer,  on  the  other  hand,  may  readily  be  able  to  reduce  the 
uncertainty.  Uncertainty  about  the  cost  of  constructing  structures  can  often  be  reduced  by  more  detailed 
estimating  techniques.  Uncertainty  about  the  years  in  which  future  floods  will  occur  cannot  be  reduced  by 
any  practical  means  and  it  is  not  worth  trying  to  do  so. 

Disagreement 

Different  people  can  view  the  same  problem  from  very  different  perspectives.  For  example,  is  the 
problem  flooding  or  is  it  developed  floodplains?  Two  experts  can  look  at  the  same  evidence  and  draw 
different  conclusions.  The  levees  have  prevented  millions  in  dollars  of  damages.  The  levees  have  destroyed 
thousands  of  acres  of  wetlands.  Which  side  is  right?  It's  quite  possible  the  two  sides  might  disagree. 

When  experts  disagree,  the  nature  of  the  disagreement  should  be  explained.  Sometimes  the 
opinions  can  be  combined  using  some  weighting  system.  This  is  usually  best  when  the  differences  are 
insignificant  for  planning  purposes.  If  the  disagreements  have  implications  for  the  planning  effort  it  is  better 
to  keep  them  separate  and  perform  a  sensitivity  analysis. 

Approximations 

Models  are  abstractions  of  real  systems.  They  inevitably  simplify  real  events  and  interdependencies. 
Tradeoffs  between  model  complexity  and  computation  costs  dictate  that  most  model  results  yield  mere 
approximations  of  the  actual  values.  One  way  to  estimate  the  uncertainty  introduced  by  the  approximation 
is  to  increase  the  resolution  of  the  model  and  measure  the  difference  between  the  two  models. 
Approximation  uncertainty  can  be  difficult  to  ascertain,  but  it  is  most  often  addressed  by  sensitivity  analysis. 

Habitat  models  are  used  to  estimate  changes  in  environmental  resources.  How  good  are  the 
approximations?  The  only  way  to  tell  is  to  compute  the  losses  with  a  better  model.  This  is  rarely  possible. 
If  a  better  model  were  available  it  would  have  been  used. 

Occasionally,  the  best  models  are  not  used  because  they  are  too  expensive.  A  WET  evaluation  may 
be  used  for  the  reconnaissance  report  when  a  HEP  evaluation  may  have  been  better.  In  a  case  like  this  the 
uncertainty  due  to  approximation  can  be  estimated  in  subsequent  studies  by  using  the  more  sophisticated 
HEP  evaluation. 
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POTENTIAL  SOURCES  OF  RISK  IN  ECOSYSTEM  RESTORATION  PLANNING 

Table  2-4  identifies  items  taken  from  EC  1105-2-206  and  offers  a  very  preliminary  attempt  by  the 
author  to  identify  the  types  of  uncertainty  that  might  be  encountered  at  each  step  in  the  planning  process.* 
The  table  offers  several  overview  broad  patterns.  First,  disagreement  would  appear  to  be  a  potential  source 
of  uncertainty  at  virtually  every  stage  of  the  planning  process.  Environmental  investment  is  a  frequently 
contested  project  purpose  attended  by  substantial  scientific  uncertainty.  The  existing  uncertainty  ensures 
that  there  is  ample  room  for  disagreement.  The  controversial  nature  of  the  plans  suggests  these  differences 
may  be  exploited  frequently.  An  open  planning  perspective  that  gives  full  voice  to  all  points  of  view  may 
be  the  best  response  to  these  differences.  From  an  analytical  process,  the  differences  found  most  worthy  of 
consideration  can  be  addressed  in  a  parametric  manner. 

Second,  there  is  a  potential  problem  with  communication.  Much  of  the  jargon  and  many  of  the 
concepts  are  new  and  still  evolving.  In  these  circumstances  there  are  likely  to  be  a  substantial  number  of 
communication  problems  among  study  stakeholders.  Information  and  education  are  the  best  ways  to  reduce 
this  type  of  uncertainty  when  it  is  encountered. 

Random  and  systematic  errors  will  be  encountered  most  often  during  the  collection  of  data  and  the 
analytical  work  of  the  study.  In  the  field  of  environmental  issues  where  ideology  often  plays  a  strong  role 
in  the  definition  of  issues  we  must  remain  aware  of  the  potential  for  systematic  errors  resulting  from  biases 
on  the  part  of  various  stakeholders.  Variability  will  likewise  be  encountered  most  often  during  the  data 
collection  tasks  where  samples  must  be  relied  on  for  information  about  the  population. 

The  random  errors  and  variability  can  be  dealt  with  through  traditional  probabilistic  and  statistical 
methods  including  Monte  Carlo  simulation,  as  appropriate.  Systematic  errors  cannot  be  addressed  by  larger 
samples  and  can  dominate  the  overall  error  in  some  cases.  The  danger  with  these  uncertainties  is  that  they 
are  often  not  recognized  and  they  go  untreated.  When  the  tools  are  poorly  calibrated  but  yield  consistent 
readings  it  is  difficult  to  recognize  the  error.  Likewise,  poorly  designed  tools  are  often  the  best  available  and 
the  error  goes  undetected.  Models  can  be  unintentionally  systematic  in  the  uncertainties  they  introduce  to 
an  analysis.  Institutional  or  human  bias  is  by  its  nature  rarely  acknowledged  and  can  be  difficult  to  detect. 
Once  detected  it  should  be  systematically  removed  from  the  data  or  planning  process. 

Unpredictability  can  be  a  problem  throughout  the  planning  process.  This  includes  those  things  we 
simply  do  not  know,  like  how  an  ecological  system  functions  or  what  the  system  would  have  been  like  had 
a  reservoir  not  been  built  60  years  ago.  It  also  includes  those  things  we  don't  know  because  we  lack  the 
expertise  needed  to  understand  them.  Some  of  this  type  of  uncertainty  can  be  removed  by  further  research  - 
more  or  better  data  or  additional  expertise.  Other  examples  of  this  uncertainty  are  irreducible  because  they 
are  fundamentally  unknowable. 


*  Both  generic  and  specific  sources  of  uncertainty  in  environmental  restoration  planning  are  covered  in  greater  detail  in 
IWR  Report  96-R-8,  An  Introduction  to  Risk  and  Uncertainty  in  the  Evaluation  of  Environmental  Investments,  March  1996. 
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TABLE  2-4:  Types  of  Uncertainty  (EC  1105-2-206) 


Random 

Systematic 

Linguistic 

Vari¬ 

Unpredict¬ 

Disagree¬ 

Error 

Error 

Imprecision 

ability 

ability 

ment 

onitormg 


&M  requirements 


ecision-making  criteria 


eport  requirements 


*  Related  to  reconnaissance  and  feasibility  phase  studies 
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With  these  overview  remarks  in  mind,  the  following  sections  suggest  how  some  of  the  different  sources  of 
uncertainty  might  arise  for  the  various  items  presented  in  Table  2-4. 

PURPOSE,  APPLICABILITY,  PHILOSOPHY,  DEFINITIONS 

The  purpose,  applicability,  philosophy  and  definitions  of  environmental  investments  are  primarily 
policy-based.  They  define  a  new  subfield  of  water  resources  planning  and  introduce  new  initiatives,  goals, 
concepts  and  language.  As  such,  there  is  bound  to  be  considerable  uncertainty  about  this  new  program 
purpose.  Even  seemingly  straightforward  terms  like  "investments"  and  "ecosystem  restoration  that  are  so 
basic  to  this  new  endeavor  are  fraught  with  uncertainty.  It  can  be  expected  that  there  will  be  confusion  and 
disagreement  about  what  all  of  this  means.  The  uncertainty  here  arises  less  from  more  classical  sources  of 
uncertainty  than  from  the  imprecision  of  language.  Ecosystem  restoration  may  conjure  visions  of  the  Garden 
of  Eden  to  some  and  a  simple  halt  to  an  imminent  loss  to  others. 

RESOURCE  SIGNIFICANCE 

Another  crucial  aspect  of  environmental  investment  planning  is  the  establishment  of  resource 
significance  needed  to  prove  a  Federal  interest  in  the  project.  There  will  be  communication  issues  about 
what  constitutes  a  resource  and  what  makes  it  significant  (linguistic  uncertainty).  The  relationships  between 
the  resources  and  project  outputs  can  be  full  of  unpredictability,  often  due  to  a  lack  of  information  and 
understanding.  As  above,  data  measurement  may  result  in  random  and  systematic  errors.  Sample  data  will 
lead  to  variability.  The  models  relating  resources  and  outputs  will  be  full  of  approximations.  Once  again, 
with  such  an  obvious  lack  of  certain  knowledge,  disagreements  over  what  constitutes  a  significant  resource 

seem  likely.’ 

NON-FEDERAL  PARTNER 

Given  the  nature  of  cost-sharing  arrangements  it  is  wise  to  expect  the  non-Federal  partner  to  give 
more  credence  to  scenarios  (i.e.,  parameter  choices  for  decision  and  value  quantities)  that  lead  to  lower  cost 
shares  for  the  non-Federal  partner.  This  may  conflict  with  other  expert  opinions  about  quantity  values. 

ECOSYSTEM  CONTEXT 

Plans  are  to  consider  the  role  of  ecological  resources  in  the  larger  context  of  the  ecosystem  and  its 
plant  and  animal  communities.  It's  not  yet  clear  that  anyone  can  do  this  in  a  deterministic  fashion.  Insofar 
as  these  appear  to  be  value  quantities  the  uncertainty  is  more  likely  to  be  systematic  than  random.  Once 
again,  language  can  be  expected  to  be  a  problem  when  the  terminology  is  so  new  and  its  precise  meaning 
so  distant.  Much  of  this  uncertainty  stems  from  a  lack  of  information  and  unpredictability.  The  models  used 


^  IWR  Report  95-R-lO  (June  1995),  Resource  Significance:  A  New  Perspective  for  Environmental  Project  Planning, 
discusses  the  concept  of  resource  significance  in  terms  of  scientific,  institutional,  and  public  criteria,  and  provides  detailed  summaries 
of  selected  programs  that  assist  in  determining  environmental  significance. 
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to  establish  the  role  of  resources  in  a  larger  context  must  be  approximate  and  disagreement  among  experts 
is  to  be  expected. 

OPERATIONAL  EFFECTIVENESS  AND  O&M  REQUIREMENTS 

The  reliability  of  environmental  investments,  i.e.,  their  ability  to  perform  the  purpose  for  which  they 
are  intended  in  a  reliable  fashion,  is  one  of  the  fundamental  uncertainties  attending  environmental  investment 
planning.  The  extent  of  required  operation  and  maintenance  activities  may  well  be  unknown  until  we 
develop  some  experience  with  ecosystem  restoration  and  other  projects. 

RECONNAISSANCE  AND  FEASIBILITY  CONSIDERATIONS 

Items  marked  with  an  (*)  in  Table  2-4  relate  to  reconnaissance  and  feasibility  phase  studies.  Most 
of  these  tasks  are  currently  part  of  existing  water  resource  planning  studies  in  the  reconnaissance  and 
feasibility  phases  of  study  and  have  been  recognized  as  exercises  in  uncertainty  for  some  time. 

The  new  nature  of  environmental  investment  planning  and  its  sometimes  hotly  contested  issues  add 
a  different  dimension  to  the  ordinary  planning  tasks.  There  may  be  greater  difficulty  in  getting  agreement 
on  such  things  as  problem  identification.  Qne  group  may  see  anything  less  than  full  restoration  as 
unresponsive.  Another  group  may  not  even  recognize  the  existence  of  a  need  for  a  contested  resource.  Both 
may  be  supported  by  the  same  data  and  different  perspectives.  At  other  times  they  may  rely  on  their  own 
data  sets.  Models  are  likely  to  be  more  approximate  because  the  science  is  not  as  developed  and  holistically 
integrated.  Disagreements  are  more  likely,  in  part  as  a  result  of  linguistic  imprecision  and  in  part  because 
of  a  realistic  basis  for  different  points  of  view. 

ADAPTIVE  MANAGEMENT  AND  MONITORING 

An  interesting  and  foresighted  concept,  adaptive  management  is  another  in  a  series  of  evolving 
concepts.  It  will  take  some  time  before  people  begin  to  get  comfortable  with  the  idea  of  adaptive 
management.  The  very  notion  of  adaptive  management  arises  from  an  enlightened  acknowledgment  of  the 
uncertainty  that  exists  in  this  planning  arena. 

In  essence,  adaptive  management  says  you  formulate  the  best  plan  possible  considering  the  range 
of  possible  results.  The  plan  is  implemented  as  formulated  but  it  is  carefully  monitored.  As  additional 
information  is  obtained  about  the  ecosystems  you  are  concerned  with  and  the  performance  of  the  project, 
you  tinker  with  the  project/  management  plan  to  adapt  to  the  new  information  you  are  gaining.  Adaptive 
management  is  learning  by  doing.  In  time  risk  and  uncertainty  analysis  will  help  planners  consider  the  range 
of  results  that  could  be  realized  from  a  unique  or  experimental  plan.  This  prior  knowledge  should  lead  to 
better  formulated  adaptive  management  plans. 
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DECISION-MAKING  CRITERIA 

Cost  effectiveness,  incremental  cost  analysis,  identifying  environmental  outputs  and  valuing  them 
where  possible  are  among  the  decision-making  criteria  that  could  be  used.  These  will  be  subjected  to  random 
and  systematic  errors  if  the  data  and  analysis  that  goes  before  them  were  subjected  to  these  errors.  Because 
the  models  used  to  generate  the  input  data  for  these  decision  criteria  are  approximations,  the  criteria  will  be 
as  well.  Disagreements  over  data,  models,  methods  and  assumptions  can  always  be  expected. 

The  criteria  supporting  plan  justification  include  acceptability,  completeness,  efficiency, 
effectiveness,  and  restoration  measure  applied.  This  adds  a  criterion  to  the  four  used  for  more  traditional 
planning  purposes.  It  also  weakens  the  efficiency  criteria  by  providing  for  a  quantitative  assessment  of  the 
costs  of  the  projects  only.  There  will  rarely  be  a  quantitative  assessment  of  the  value  of  the  outputs  produced 
by  the  plan.  This  makes  the  selection  of  an  efficient  project  even  more  uncertain  than  usual. 


REVIEW  OF  EXISTING  CORPS  RISK  ANALYSIS  GUIDANCE  AND  REPORTS 

The  preceding  sections  presented  a  discussion  of  the  types  of  quantities  that  can  be  uncertain  as  well 
as  the  nature  of  the  different  kinds  of  uncertainty  that  are  most  commonly  found  with  empirical  quantities. 
In  an  abbreviated  review  of  some  environmental  investment  planning  issues  we  have  attempted  to  identify 
some  of  the  potential  sources  of  uncertainty  in  primarily,  but  not  entirely,  empirical  quantities  that  will  be 
encountered  in  environmental  investment  planning. 

With  the  different  types  of  quantities  that  can  be  uncertain  in  mind,  the  types  of  uncertainty  and  a 
general  idea  of  where  uncertainty  may  be  encountered  in  environmental  resource  planning,  we  can  begin  to 
consider  existing  Corps  guidance  on  risk  and  uncertainty  analysis  to  learn  if  it  may  be  helpful  in  the 
development  of  risk  analysis  guidance  for  environmental  investment  planning. 

In  the  following  sections  selected  existing  risk  analysis  guidance  used  by  the  Corps  of  Engineers  are 
reviewed.  The  review  begins  with  a  complete  citation  of  the  guidance  and  proceeds  in  a  narrative  fashion 
to  describe  the  applicability  of  that  guidance  to  the  development  of  risk  analysis  guidance  for  environmental 
investment  planning.  Chapter  Two  of  this  report  concludes  with  a  summary  discussion  of  the  guidance 
reviewed. 


Engineer  Circular  1105-2-205  Risk  Based  Analysis  for  Evaluation  of  Hvdrolosv/Hvdraulics  and 
Economics  in  Flood  Damage  Reduction  Studies 

The  purpose  of  the  EC  is  to  provide  "...guidance  on  the  evaluation  framework  to  be  used  in  Corps 
of  Engineers  flood  control  and  flood  damage  reduction  studies."  The  EC  itself  is  four  pages  in  length  and 
is  accompanied  by  54  pages  of  extended  examples  in  three  appendices.  This  EC  represents  detailed,  specific, 
and  directed  guidance  for  field  personnel.  It  is  an  excellent  example  of  directive  guidance. 
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The  EC's  Background  material  is  as  applicable  for  environmental  investment  planning  as  it  is  for 
flood  damage  reduction  studies.  It  states  that  risk  and  uncertainty  are  intrinsic  in  water  resources  planning 
issues;  best  estimates  of  key  variables  and  sensitivity  analysis  are  to  be  used;  and,  risk  analysis  can  be 
advantageous  to  the  planning  process.  These  are  important  principles,  well  worth  repeating  for  any  planning 
context. 


The  structure  of  the  guidance  {Purpose,  Applicability,  References,  Background,  Definitions,  Key 
Variables  in  a  Risk-Based  Analysis,  Policy  and  Required  Procedures,  and  Examples  of  Risk-Based  Analysis) 
provides  an  excellent  model  for  future  detailed  guidance.  The  definitions  are  important  for  limiting 
uncertainty  that  results  from  an  imprecise  language.  The  emphasis  placed  on  key  variables  illustrates  the 
importance  of  this  task.  It  is  perhaps  the  single-most  important  first  step  on  which  any  environmental 
investment  planning  risk  analysis  should  focus.  Preliminary  identification  of  the  key  variables  is  essential 
so  that  the  nature  of  the  uncertainty  can  be  discerned  and  potential  techniques  for  addressing  it  developed. 

The  details  in  the  Policy  and  Required  Procedures  are  too  specific  to  the  estimation  of  flood  damage 
reduction  expected  annual  damages  to  warrant  any  detailed  review  here.  Once  again,  however,  the  EC 
provides  a  valuable  model  for  future  detailed  guidance.  There  is  a  clear  directive  to  use  risk-based  analysis 
followed  by  a  statement  of  the  ultimate  goal  of  that  analysis  that  includes  a  statement  of  the  minimal 
requirements  for  a  risk  analysis.  The  EC  prescribes  that  the  results  of  the  analysis  present  the  expected  value 
and  the  distribution  of  NED  benefits.  Considerable  attention  is  given  to  the  issue  of  how  to  handle  what  was 
once  called  "freeboard". 

The  Appendices^  particularly  provide  excellent  examples  of  possible  directions  for  any 

future  environmental  investment  planning  program  risk  analysis  guidance.  It  also  provides  some  examples 
of  what  to  avoid. 

Appendix  A  explains  the  nature  of  the  plan  formulation  problem  for  flood  damage  reduction  studies 
in  a  coherent,  overview  fashion.  This  "big-picture"  view  of  the  problem  is  an  essential  starting  point.  In 
environmental  investment  planning  a  similar  effort  will  be  needed  to  tie  the  uncertainties  inherent  in  the 
biological  models  with  the  engineering  and  economic  uncertainty.  Environmental  planning  efforts  are  too 
unique  and  varied  to  lend  themselves  easily  to  representation  by  any  one  simple  model.  Nonetheless,  a 
generic  model  of  the  basic  elements  of  the  decision  problem  would  be  a  unifying  structure  for  analysts  to 
formulate  their  risk  analysis  about.  A  framework  that  is  a  corollary  to  the  hydroeconomic  model  of  Figure 
A-1  in  the  EC,  showing  how  the  major  components  of  the  analysis  relate  to  one  another,  would  be  an 
invaluable  aid  to  the  development  of  future  risk  analysis  procedures  and  guidance. 

The  “Risk-Based  Strategy  for  Project  Formulation”  section  in  the  EC  is  a  good  idea  in  principle. 
The  disadvantage  of  this  and  other  sections  of  the  Appendix  A  example  for  environmental  investment 
planning  is  that  it  relies  more  on  the  Monte  Carlo  process  and  simulation  than  the  risk  and  uncertainty 
analyses  for  environmental  investment  planning  might.  Section  II  and  subsequent  sections  of  the  Appendix 
provide  detailed  discussions  of  the  uncertainty  attending  relationships  presented  in  the  basic  risk-analysis 
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problem.  This  format  is  a  good  one.  It  would  be  appropriate  to  discuss  habitat  model  uncertainty, 
environmental  output  uncertainty,  structure  performance  uncertainty,  and  cost  uncertainty  separately  when, 
and  if,  guidance  is  developed  for  environmental  risk  analysis.  The  structure  used  in  this  EC  is  exceptional. 

While  much  of  the  content  of  the  EC  is  far  too  specific  to  flood  damage  reduction  to  be  usefully 
reviewed  here,  there  is  much  of  general  interest.  Section  III  includes  a  section  titled  Stage  Uncertainty 
Factors  and  Issues”,  which  provides  a  list  of  the  specific  quantities  that  can  give  rise  to  different  types  of 
uncertainty.  Table  A-4  in  Section  III  does  the  same  in  more  detail.  One  option  is  to  use  and  build  upon  the 
presentation  of  Table  A-4,  incorporating  the  structure  provided  by  Morgan  and  Henrion's  (1992)  taxonomy. 

The  far  right  column  of  Table  A-4  provides  a  somewhat  comprehensive  list  of  key  variables.'®  We 
think  two  additional  columns  identifying  the  type  of  quantity  and  the  source  of  uncertainty  in  the  context 
established  by  Morgan  and  Henrion  (1992)  presented  earlier  in  this  report  would  be  useful.  With  this 
additional  data  we  would  have  a  better  idea  how  we  might  appropriately  go  about  addressing  the  uncertainty 
inherent  in  these  key  variables. 


a,nH^hnnk  for  Risk  Perception  and  Communication  in  Water  Resources  Plannins,  Part  I  Underpinninss 
and  Planning  Annlicationx  and  Part  II  An  Annotated  Bibliography,  IWR  Reports  93-R-13  and  14, 
October  1993 

The  Guidebook  provides  a  review  of  the  major  observations  and  theories  found  in  the  literature  about 
how  people  perceive  risk.  It  sets  forth  the  maximization  of  expected  utility  as  the  normative  model,  then 
proceeds  to  identify  the  limitations  to  this  model.  In  subsequent  chapters  the  guidebook  develops  the 
limitations  to  the  normative  model  in  substantial  detail.  The  limitations  discussed  include:  the  filters  that 
risk  information  must  penetrate;  cognitive  problems  with  risky  decisions;  and  the  role  of  social  pressure  and 
amplification  in  communication. 

The  major  contribution  of  the  guidebook  is  Chapter  VI,  which  presents  guidelines  intended  to  assist 
water  resource  planners  and  managers  in  their  efforts  to  communicate  with  the  public  and  decision-makers 
about  situations  in  which  risk  is  important. 

While  there  is  little  in  the  Guidebook  that  is  specific  to  environmental  investment  planning,  there 
is  a  great  deal  that  could  be  useful  in  minimizing  or  eliminating  the  uncertainty  that  results  from  linguistic 


This  list  provides  a  good  example  of  one  of  our  most  serious  concerns  about  such  lists.  Field  personnel 
looking  for  direction  that  will  help  them  satisfy  the  analytical  demands  of  their  higher  authority  can  tend  to  take  ECs  as 
gospel  rather  than  guidance.  Lists  can  be  dangerous  when  they  are  not  comprehensive  and  it  is  very  difficult  to  be 
comprehensive.  For  example,  personal  experience  with  sensitivity  studies  has  indicated  that  the  lowest  ground  elevation 
at  which  flood  damages  begin,  sometimes  called  the  zero  damage  point  or  ground  elevation,  is  the  single  most  important 
variable  in  expected  annual  damage  calculations  in  areas  with  frequent  flooding.  This  variable  does  not  appear  in  the  list 
and  may  never  be  investigated. 
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imprecision  and  disagreement,  two  likely,  common  sources  of  uncertainty  in  the  environmental  investment 
planning  program.  Communication  is  the  key  to  addressing  both  these  sources  of  uncertainty.  The 
guidebook  presents  a  set  of  guiding  principles  when  the  objective  is  to  get  people  to  understand  an  uncertain 
situation  and  another  set  of  principles  when  getting  people  to  take  action  is  the  goal.  These  principles  are 
reproduced  in  Figure  2-1. 

Though  the  orientation  of  the  Guidebook  seems  to  be  toward  communicating  physical  risks  to  the 
public,  there  is  much  insight  that  is  useful  to  anyone  involved  in  risk  analysis.  It  describes  such  common 
phenomena  as  denial,  which  is  frequently  encountered  in  environmental  studies.  "The  method  chosen  for 
coping  with  uncertainty  in  many  planning  and  policy-making  contexts  is  simple  if  not  very  defensible:  those 
involved  pretend  it  does  not  exist"  (p.l).  About  environmental  management,  the  Guidebook  says, 
"...ecological-modeling  capability  may  reflect  substantial  ignorance  of  how  relevant  systems,  such  as 
wetlands,  actually  function  in  the  larger  context,  as  well  as  a  serious  lack  of  historical  data  on  inputs  and  end 
points"  (p.  3).  In  presenting  examples  of  problems  public  agencies  face  that  do  not  fit  the  normative  model, 
the  guidebook  provides  useful  insight  into  environmental  risk  analysis  in  the  following  quotation: 

"Ultimately,  the  Secretary  of  the  Army's  position  on  the  permit  will  have  to  be  based,  in 
part,  on  the  expert  judgment  of  ecologists  about  the  probabilities  of  various  types  and  levels 
of  harm.  But  there  is  no  requirement  that  even  entirely  rational  citizens  should  accept  these 
judgments." 

This  last  point  is  an  excellent  example  of  the  need  for  clear  and  concise  communication  about  all  matters 
of  risk  and  uncertainty,  because  they  are  a  hard  sell  under  the  best  of  circumstances. 

While  the  Guidebook  is  full  of  useful  insights,  examples  and  suggestions,  there  appears  to  be  little 
that  could  be  explicitly  adapted  from  the  Guidebook  io  draft  risk  guidance  for  environmental  investments. 
The  annotated  bibliography  that  constitutes  Part  II  of  the  Guidebook  was  used  in  the  preparation  of  the 
bibliography  that  accompanies  the  literature  review  in  Chapter  Three  of  this  report. 
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FIGURE  2-1:  Guiding  Principles  in  Communicating  Uncertainty 
When  Understanding  the  Uncertainty  Is  the  Objective 

•  Uncertainty  should  be  expressed  in  a  variety  of  ways.  Care  should  be  taken  to  avoid 
encouraging  thinking  of  independent  events  as  cyclic,  with  fixed  return  intervals. 

•  Disagreements  among  experts  should  be  made  explicit  and  not  concealed.  If  possible,  the  range 
of  options  should  have  probability  weights  attached  to  alternative  possibilities. 

•  The  decision  problem  outcomes  should  be  framed  in  at  least  two  ways,  one  stated  as  losses  from 
best  case  and  one  as  gains  from  worst  case. 

•  Stress  the  analogy  of  the  project  to  an  insurance  policy  against  catastrophic  loss  whenever  this  is 
appropriate. 

•  Provide  information  that  allows  the  audience  to  assess  the  risk  of  the  contemplated  project 
relative  to  other  activities  and  programs,  both  individual  and  collective. 

When  the  Objective  is  to  Take  Action  About  the  Uncertainty 

•  The  campaign  should  effectively  convey  a  message  about  the  seriousness  of  the  risk. 

•  The  program  should  provide  social  reinforcement  of  risk  reduction  behavior  especially  at  the 
local  level. 

•  The  campaign  should  make  an  attempt  to  convince  the  consumers  that  their  actions  aimed  at 
reducing  risk  will  help  to  mitigate  risk  impacts. 

•  Risk  Reduction  efforts  requested  by  the  campaign  should  be  equitable. 

•  The  specific  strategies  of  the  campaign  should  rely,  to  the  extent  possible,  on  providing  a 
feedback  on  risk  reduction  efforts  and  providing  economic  and  social  incentives  for  doing  so. 

Note:  Adapted  from  Summary  p.  73. 


Guidelines  for  Risk  and  Uncertainty  Analysis  in  Water  Resources  Plannins  Volume  I  Principles  with 
Technical  Appendices.  IWR  Report  92-R-l,  March  1992 

Volume  I  oiXhQ  Guidelines  provides  a  discussion  of  basic  principles  of  risk  and  uncertainty  analysis 
that  provide  a  relevant  and  concise  context  for  the  development  of  risk  analysis  principles  for  environmental 
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investment  planning.  The  basic  concepts  presented  in  Chapter  2  are  somewhat  dated  in  that  there  is  now  a 
widely  accepted  interchangeable  use  of  the  terms  uncertainty  and  risk,  a  position  not  taken  in  the  March  1992 
report.  Uncertainty  is  used  as  the  comprehensive  term  that  describes  situations  of  risk  (i.e.,  an  unwanted 
consequence  and  its  probability  of  occurring),  as  well  as  the  full  range  of  uncertainties  discussed  earlier  in 
this  report. 

Risk  analysis  is  described  as  consisting  of  an  assessment  and  a  management  function.  The 
distinction  between  the  assessment  of  risk,  the  technical  analysis  required  to  develop  results,  and  the 
management  of  risk,  what  decision-makers  do  with  those  uncertain  results,  is  still  a  valuable  one.  Any  risk 
analysis  guidance  should  pay  particular  attention  to  the  duality  of  the  risk  analysis.  It  consists  of  an 
assessment  stage  that  includes  technical  analysis  and  a  management  stage  that  includes  the  evaluation  of  the 
results  of  the  assessment  in  the  broader  planning  context.  These  two  components  will  be  substantially 
different  in  terms  of  their  foci,  stakeholders  and  techniques.  For  example,  model  uncertainty  may  be  a 
substantial  assessment  issue,  while  expert  disagreement  could  dominate  the  risk  management  phase  of  the 
analysis. 

Chapter  3  on  “Risk  and  Uncertainty  Analysis  in  the  Planning  Process”  provides  a  good  starting  point 
for  any  new  risk  analysis  guidance.  If  the  initial  guidance  is  to  avoid  reliance  on  specific  techniques  and 
models  it  should,  at  a  minimum,  provide  guidance  in  the  explicit  context  of  the  environmental  investment 
planning  process. 

Suggestions  that  multiple  “without  project  condition”  scenarios  be  developed,  that  key  variables  at 
each  stage  of  the  planning  process  be  identified,  and  that  the  risk  analysis  transition  gradually  through  the 
planning  process  from  an  emphasis  on  assessment  to  an  emphasis  on  management  are  as  relevant  for 
environmental  investment  planning  guidance  as  they  are  for  the  general  planning  process.  The  guidebook 
stresses  the  need  to  be  open  and  above  board  about  the  uncertainty  analysis.  This  is  even  more  important 
with  ecological  uncertainty. 

Chapter  5  on  “Techniques  for  Dealing  with  Risk  and  Uncertainty”  is  particularly  important  because 
it  presents  some  of  the  range  of  techniques  that  can  be  used  in  uncertainty  analysis.  Ecological  restoration 
planning  has  the  opportunity  to  embrace  a  broader  range  of  analytical  tools  than  have  been  adopted  so  far. 
Flood  damage  reduction  studies  and  lock  and  dam  rehabilitation  studies  have  appropriately  emphasized 
simulations.  Environmental  investment  planning  may  require  a  broader  range  of  tools.  This  chapter 
provides  a  good  place  to  start  identifying  possible  tools. 

Chapter  6  on  risk  communication  has  been  more  fully  developed  in  IWR  Reports  93-R-13  and  14. 
Nonetheless,  some  useful  distinctions  are  made  between  internal  and  external  communication  of  risk  issues. 
The  chapter  provides  a  good  summary  of  the  most  significant  risk  communication  issues. 

The  outline  presented  in  Chapter  7,  “A  Risk  and  Uncertainty  Analysis  Outline”,  provides  a  useful 
model  that  might  be  emulated  by  those  doing  environmental  investment  planning.  The  chapter  makes  a 
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fundamental  point  that  today's  analysts  must  understand:  "There  is  no  formula  for  conducting  a  risk  and 
uncertainty  analysis."  Even  so,  a  systematic  approach  to  any  risk  and  uncertainty  analysis  can  and  should 
be  developed  prior  to  the  initiation  of  the  study.  The  outline  presented  in  the  chapter  presents  a  viable 
example  of  how  this  might  be  done. 

The  Technical  Appendices  of  Volume  I  provide  considerable  useful  information  for  those  unfamiliar 
with  the  basic  tenets  of  risk  analysis.  The  bibliography  in  Appendix  A  provides  an  excellent  cross-sectional 
summary  of  the  general  risk  literature  prior  to  1992.  Little  of  direct  use  to  those  interested  in  explicit 
applications  of  risk  analysis  techniques  to  environmental  issues  will  be  found  in  these  volumes,  however. 

Excerpts  from  the  Economic  and  Environmental  Principles  and  Guidelines  for  Water  and  Related 
Land  Resources  Implementation  Studies  (also  called  the  “Principles  and  Guidelines  or  P&G  ),  relating  to 
risk  and  uncertainty  are  reproduced  in  Appendix  B. 

The  basic  concepts  of  probability  are  presented  in  Appendix  C.  The  presentation  is  conventional 
and  a  bit  dry.  It  would  provide  an  adequate  review  of  anyone  seeking  a  concise  brush-up  on  principles  of 
probability  but  offers  little  real  insight  to  those  who  are  far  removed  in  time  from  their  formal  training  in 
probability. 

Appendix  D,  “Distributions”,  provides  a  handy  overview  of  several  key  aspects  of  probability 
distributions.  Following  some  rather  stiff  definitions  of  random  variables,  probability  density  functions 
(PDF),  and  cumulative  distribution  functions  (CDF),  the  appendix  contributes  some  useful  insights  for  the 
non-statisticians.  The  relationship  between  the  PDF  and  CDF  is  developed  so  the  reader  can  understand  that 
there  are  different  ways  of  presenting  the  same  information.  The  appendix  pays  attention  to  the  role  of 
parameters  in  determining  the  shapes  and  locations  of  distributions  and  this  too  is  useful  to  new  risk  analysts. 
The  appendix  concludes  with  a  discussion  of  distribution  fitting  that  provides  a  good  introduction  to  the 
topic.  Understanding  probabilities  and  distributions  is  essential  for  any  risk  analysis,  whether  assessment 
or  management,  and  Appendices  C  and  D  are  helpful  starting  points  for  this  purpose. 

Appendix  E,  “Sampling  Techniques”,  addresses  a  useful  topic.  Though  it  presents  a  useful 
introduction  to  concepts  like  simple  random  samples,  stratified  random  samples,  and  cluster  samples,  it  is 
far  too  superficial  to  be  very  useful. 

Expected  utility  theory  is  addressed  in  Appendix  F.  This  is  a  very  important  concept  in  the  field  of 
risk  analysis.  The  differences  among  risk-adverse,  risk-neutral,  and  risk-seeking  attitudes  are  illustrated  with 
a  flood  damage  reduction  example.  These  risk  attitudes  are  essential  to  understand  for  the  risk  management 
phase  of  a  risk  analysis. 

Risk-adverse  people  tend  to  be  willing  to  pay  more  than  the  expected  damages  they  could  incur  in 
order  to  eliminate  the  possibility  of  those  damages  destroying  their  property  or  security.  For  example,  we 
might  pay  $300  annually  in  fire  insurance  to  avoid  a  fire  loss  with  an  expected  value  of  $50  per  year.  A  risk 
neutral  person  would  only  pay  $50  to  avoid  this  fire  risk.  A  risk-seeking  individual  would  pay  less  than  $50 
to  avoid  the  potential  fire  loss.  These  are  concepts  risk  analysts  need  to  understand  to  access  the  risk 
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literature  and  to  address  people's  perceptions  of  risky  situations.  The  appendix  also  presents  a  couple  of 
alternatives  to  the  expected  utility  criteria  forjudging  risks. . 

The  forecasting  overview  in  Appendix  G  provides  a  reasonable  overview  of  the  components  of  time 
series  data  and  some  simple  time  series  forecasting  methods.  The  value  of  this  appendix  lies  in  the  range 
of  topics  it  covers,  including  qualitative  forecasting  methods  which  may  be  very  important  for  environmental 
investment  planning. 

Simulation  is  the  topic  of  Appendix  H.  Brief  discussions  of  when  to  simulate  as  well  as  the 
advantages  and  disadvantages  of  simulation  are  presented.  A  cursory  overview  of  the  Monte  Carlo  process 
is  provided.  The  material  in  this  appendix  is  most  rudimentary.  Many  of  the  simulation  languages  cited  are 
now  defunct.  There  are  far  better  introductions  to  the  Monte  Carlo  process  contained  in  the  users  manuals 
of  @RISK  and  Crystal  Ball,  two  commercially  available  products. 

Appendix  I,  “Decision-Making  Under  Uncertainty”,  is  a  useful  reference  to  those  new  to  the  field 
of  risk  analysis.  The  decision  criteria  presented  in  this  appendix  include:  partial  probabilities,  LaPlace 
criterion,  maximin  criterion,  maximax  criterion,  dominance  criterion,  Hurwicz  criterion,  and  the  minimax 
criterion.  One  or  more  of  these  criteria  may  be  very  useful  to  the  risk  management  stage  of  a  risk  analysis 
because  they  are  simple  methods  of  addressing  multi-criteria  decision  methods.  The  criteria  allow  analysts 
or  decision-makers  to  make  value  judgments  about  the  uncertainty  faced  in  the  evaluation  of  project 
outcomes.  Inasmuch  as  traditional  benefit-cost  analysis  will  not  be  practical  in  many  environmental 
investment  planning  efforts,  alternative  decision  criteria  may  be  of  considerable  interest. 

Appendix  J  presents  a  simple  introduction  to  the  most  basic  notions  of  Bayesian  inference.  The 
example  presented  provides  an  excellent  introduction  to  the  notion  of  how  subjective  probabilities  can  be 
incorporated  into  the  assessment  phase  of  a  risk  analysis.  The  danger  of  this  example,  however,  is  that  it 
could  leave  the  reader  seriously  underestimating  the  complexity  of  more  sophisticated  Bayesian  analysis. 
Bayesian  analysis  is  a  technique  that  could  be  expected  to  be  very  useful  in  and  important  to  risk  analyses 
of  environmental  issues.  Generally,  these  techniques  are  beyond  the  capability  of  most  field  personnel  and 
require  the  support  of  experts.  Generally,  we  assume  no  one  would  pay  $500  for  a  $300  television.  Why 
pay  more  than  something  is  worth?  This  reasoning  serves  us  well  in  most  circumstances.  However,  in 
situations  involving  risk  this  reasoning  may  no  longer  be  useful.  The  insurance  industry  is  able  to  function 
because  people  are  willing  to  pay  more  in  premiums  than  the  damages  they  as  a  group  will  incur  over  the 
long-run. 


Guidelines  for  Risk  and  Uncertainty  Analysis  in  Water  Resources  Planning  Volume  II  Examples.  IWR 
Report  92-R-2,  March  1992 

This  report  presents  hypothetical  example  risk  analyses  for  a  flood  control  and  a  navigation  project. 
The  examples  are  replete  with  helpful  examples  of  how  to  incorporate  risk  analysis  into  the  traditional 
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planning  process.  The  examples  range  from  suggestions  about  how  to  remove  the  guise  of  determinant 
knowledge  from  the  report  language  to  the  first  developed  example  of  how  uncertainty  can  be  incorporated 
into  the  estimation  of  benefits. 

The  examples  in  this  volume  were  seminal  works  but  they  have  been  largely  surpassed  by  more 
recent  work.  However,  detailed  examples  such  as  these  serve  an  extremely  useful  purpose  insofar  as  they 
give  agency  personnel  a  concrete  example  of  how  the  application  of  risk  analysis  might  proceed.  They  also 
provide  a  platform  from  which  refinements  of  the  analysis  can  proceed.  Developing  examples  that  illustrate 
how  risk  analysis  can  be  incorporated  into  environmental  investment  planning  studies  would  be  a  most  useful 
future  exercise. 


Guidance  for  Major  Rehahilitation  Evaluation  Reports  Fiscal  Year  199X,  annual 

This  annual  guidance  memorandum  provides  general  guidance  on  the  preparation  of  major 
rehabilitation  evaluation  reports  beginning  in  fiscal  year  1995.  As  such,  it  is  broader  in  scope  than  the  other 
risk-analysis  guidance  reviewed  in  this  report.  For  the  most  part,  there  is  little  of  direct  interest  in  this  annual 
memorandum  to  the  environmental  investment  program.  The  memo  addresses  major  rehabilitations  of 
hydropower  and  navigation  projects  with  an  emphasis  on  reliability  analysis.  Nevertheless,  there  are  some 
terms  and  concepts  of  general  interest  in  risk  analysis. 

The  guidance  distinguishes  major  rehabilitations  for  reliability  from  those  done  for  improved 
efficiency.  It  is  significant  that  considerable  care  is  taken  in  defining  these  two  mutually  exclusive 
categories  of  major  rehabilitation.  There  is  a  clear  intention  to  try  to  avoid  uncertainty  that  stems  form 
imprecise  language  in  the  definitions  of  terms.  This  is  a  virtue  to  be  emulated. 

The  guidance  provides  a  somewhat  detailed  description  of  the  contents  of  a  major  rehabilitation 
evaluation  report  that  would  appear  to  be  helpful  to  field  personnel.  An  outline  of  an  environmental 
investment  planning  report  could  be  similarly  useful  in  that  it  would  provide  structure  without  constraining 
creativity. 

The  document  defines  the  term  "base  condition"  as  the  alternative  which  all  other  plans  will  be 
measured  against.  It  goes  on  to  offer  some  assumptions  implied  by  the  term.  Because  they  are  unique  to 
the  major  rehabilitation  program  they  will  not  be  reviewed  here.  Base  condition  is  a  term  used  in  place  of 
"without  project  condition"  in  the  major  rehabilitation  planning  context.  We  suggest  it  may  be  a  term  readily 
adapted  to  the  environmental  investment  program  where  issues  like  ecosystem  restoration  make  without 
project  and  with  project  terminology  rather  confounding. 

The  steps  identified  for  portraying  the  base  condition  recommend  the  use  of  event  trees  and 
analytical  solutions  to  problems  in  deference  to  the  automatic  use  of  simulations.  This  is  an  encouraging 
step  inasmuch  as  it  attempts  to  broaden  the  array  of  risk  analysis  tools  used. 
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The  document  will  be  a  useful  resource  for  the  development  of  guidance  relating  to  "failure" 
scenarios.  The  adaptive  management  framework  recommended  for  ecosystem  restoration  recognizes  the 
considerable  uncertainty  that  currently  attends  our  knowledge  in  this  area.  One  of  the  early  concerns  of 
environmental  investment  planners  is  how  will  different  projects  perform?  Will  they  succeed  or  will  they 
fail? 


This  annual  memorandum  provides  a  failure  scenario  framework  that  may  serve  as  a  starting  point 
for  risk  analysis.  The  base  condition  must  address  the  timing,  frequency,  and  consequences  of  system 
disruption.  All  of  these  are  unknown  and  must  be  estimated.  The  steps  showing  how  this  can  be  done  for 
rehabilitation  of  projects  can  be  readily  adapted  for  other  failure  scenarios.  To  the  base  condition  several 
alternative  plans  are  compared.  Instead  of  the  "with  rehabilitation  condition"  of  this  guidance  an  alternative 
plan  scenario  might  be  used  and  the  comparison  could  proceed  conceptually  as  laid  out  in  the  guidance 
document. 

Enclosure  1  presents  a  conceptual  approach  to  rehabilitation  that  introduces  the  useful  idea  of  time 
paths.  In  two-dimensional  space  a  time  path  shows  how  some  variable  of  interest,  usually  benefits  or  costs, 
vary  over  a  period  of  time,  usually  a  planning  horizon.  These  ideas  could  easily  be  adapted  for  population, 
water  quality,  or  other  environmental  values.  Figure  2  on  page  18  presents  a  probabilistic  view  of  time  paths 
that  shows  how  means  and  variances  change  over  time.  It  is  a  well  done  example  that  could  be  readily 
adapted  for  other  contexts.  The  time  path  device  is  particularly  useful  for  environmental  investment 
planning  because  it  enables  the  analyst  to  represent  the  dynamic  and  stochastic  nature  of  key  quantities. 

Enclosure  2  provides  an  introduction  to  the  assessment  of  structural  reliability.  As  presented,  the 
concept  holds  little  immediate  interest  for  environmental  investment  planners.  However,  there  may  be 
reason  to  investigate  reliability  theory,  beginning  with  such  basic  notions  as  these  to  evaluate  the 
performance  of  environmental  alternatives. 

The  basic  notion  behind  the  reliability  analysis  is  that  of  a  safety  factor.  Safety  factors  are  expressed 
as  the  ratio  of  capacity  to  demand.  Perhaps  the  safety  factor  notion  can  be  adapted  for  environmental  use. 
For  example,  if  an  acre  has  vegetative  cover  (capacity)  for  10  organisms  and  two  live  on  the  acre  (demand), 
the  "safety  factor"  is  five.  In  Enclosure  2  a  probabilistic  approach  is  used  to  incorporate  uncertainty  into  the 
development  of  safety  factors. 

Enclosure  3  is  an  overview  of  hydropower  and  navigation  benefits  of  no  interest  to  this  report. 
Enclosure  4  ,  however,  shows  an  example  of  how  to  calculate  an  expected  annual  value  that  could  be  a  very 
helpful  reference  for  analysts.  Virtually  all  examples  of  expected  annual  value  computations  come  from 
flood  damage  reduction  studies.  Enclosure  4  shows  examples  of  expected  annual  costs  and  benefits  based 
on  alternative  streams  of  benefits  and  costs  over  a  10-year  period.  While  the  example  is  good  it  does  not  take 
full  advantage  of  the  opportunity  to  instruct  analysts  with  less  experience  with  risk  analysis.  For  example, 
Table  2a  shows  the  computation  of  marginal  probabilities,  an  important  concept  for  environmental  risk 
analysis.  Unfortunately,  many  readers  will  be  unable  to  follow  the  example.  A  simple  explanation  that  the 
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probability  of  failure  is  0.1  in  the  first  year,  0.1 1  in  the  second  year,  0.121  in  the  third,  0.133  in  the  fourth, 
0.146  in  the  fifth,  and  so  on,  would  help.  The  example  provides  all  the  information  needed  to  figure  this 
out  in  Table  1  but  relatively  few  will  do  so  with  ease.  The  series  presented  has  a  failure  probability  that 
begins  at  0.1  and  grows  at  10%  a  year.  The  probability  of  success  is,  then,  one  minus  the  probability  of 
failure.  For  the  first  five  years  these  probabilities  are  0.9,  0.89,  0.879,  0.867,  and  0.854.  While  we  do  not 
suggest  this  is  an  adequate  explanation,  we  do  suggest  explanations  such  as  this  and  more  are  critically 
important  if  the  risk  analysis  guidance  is  to  help  analysts  rather  than  intimidate  or  frustrate  them. 


Engineer  Technical  T.etter  1110-2-328  Reliability  Assessment  of  Existins  Levees  for  Benefit 
Determination.  22  March  1993 

The  purpose  of  the  ETL  is  to  provide  "...guidance  on  the  assessment  of  the  structural  reliability  of 
existing  levees,  primarily  for  the  purpose  of  estimating  flood  damages  for  the  "without  project  condition... 
The  detailed  and  technical  guidance  provided  in  this  ETL  provides  an  excellent  example  to  which  potential 
risk  analysis  guidance  for  the  environmental  Investment  planning  program  can  aspire.  Assessing  the 
reliability  of  existing  levees  is  the  raison  d'etre  for  this  ETL.  Assessing  the  reliability  of  structures  and  other 
management  measures  is  likely  to  become  an  important  consideration  in  an  adaptive  management  approach 
to  environmental  planning.  Likewise,  assessing  the  reliability  of  projects  to  which  impacts  are  linked  may 
become  a  significant  consideration.  In  either  or  both  cases  explicit  guidance  like  that  in  the  ETL  may  be 
most  helpful. 

The  ETL  directs  that  the  without  project  condition  "...be  based  on  what  is  most  likely  to  occur  if  the 
project  is  not  built".  Basing  without  project  conditions  on  what  is  most  likely  to  happen  in  actuality  is  a 
significant  step  forward  in  the  planning  process  when  compared  to  the  prior  method  of  relying  on 
engineering  or  policy  standards."  Future  risk  analysis  guidance  should  adhere  to  the  actual  practice  standard 
whenever  possible. 

The  ETL  provides  a  reasonable  model  to  emulate.  The  procedure  for  conducting  the  evaluation  of 
existing  levee  reliability  is  outlined  in  several  generic  steps  beginning  in  paragraph  five.  Identifying  the 
known  major  steps  required  for  an  analysis  is  useful.  The  ETL  directs  that  these  steps  be:  a)  The 
Preliminary  Field  Survey;  b)  Slope  Stability  Analysis;  and,  c)  Past  Performance.  The  procedure  goes  on  to 
address  the  means  by  which  professional  disagreements  will  be  resolved:  the  Joint  Field  Survey.  It  is 
significant  and  farsighted  that  the  guidance  recognizes  expert  disagreement  as  an  expected  and  reasonable 
source  of  uncertainty.  The  guidance  provides  for  an  orderly  resolution  of  such  issues  by  directing  an 
inclusive  process  to  resolve  these  differences.  We  think  risk  analysis  in  the  environmental  investment 
planning  program,  where  expert  disagreement  and  controversy  can  be  expected,  would  be  well  advised  to 
emulate  the  spirit  of  this  ETL. 


’ '  For  example,  past  policy  for  levees  included  ignoring  benefits  provided  by  substandard  levees  even  when 
benefits  were  known  to  have  accrued.  Other  examples  include  the  prior  reliance  on  standards  for  freeboard  for  levees  and 
flood  walls  and  underkeel  clearance  for  navigation  channels. 
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A  second  approach,  the  Template  Method,  is  also  offered  in  the  guidance  as  an  alternative  to  the 
model  of  items  a  through  c  above.  Though  the  second  method  is  offered  for  use  when  data  for  the  preferred 
method  are  unavailable,  the  fact  that  alternative  methods  are  presented  is  desirable.  When  environmental 
risk  analyses  confront  model  uncertainty,  it  is  advisable  that  guidance  allow  and  even  encourage  alternative 
analytical  models  and  risk  models  until  a  broad  based  consensus  on  the  relative  merits  of  various  models 
begins  to  build  among  the  environmental  community. 

The  procedure  guidance  concludes  with  a  straightforward  statement  about  the  applicability  of  the 
ETL.  Attachments  to  the  ETL  provide  effective  illustration  of  the  template  method  and  the  procedure 
described  above. 

On  balance,  the  ETL  provides  a  good  model  of  the  conciseness  and  clarity  that  future  detailed 
guidance  should  aspire  to.  The  guidance  builds  in  flexibility  and  an  acknowledgment  of  expert  disagreement 
and  a  reliance  on  actual  practice  that  may  prove  to  be  useful  themes  for  any  future  risk  analysis  guidance. 


Engineer  Technical  Letter  1110-2-XXXX  Uncertainty  Estimates  for  Non-Analvtic  Frequency  Curves. 
November  1993 


This  draft  ETL  presents  a  methodology  and  computer  program  for  computing  the  uncertainty  about 
non-analytic  frequency  curves.  It  provides  an  excellent  example  of  the  type  of  guidance  that  is  appropriate 
once  an  integrated  model  has  been  developed.  To  the  extent  that  flow-frequency  information  becomes 
important  in  environmental  resource  planning  studies,  this  guidance  could  be  useful,  if  it  is  implemented  in 
its  current  form. 

In  hydrologic  studies  analysts  either  use  an  analytic  distribution  of  flows,  like  the  log-Pierson  III, 
or  they  use  non-analytic  techniques,  like  graphical  fits  of  ordered  data  points  to  a  curve.  This  latter  technique 
is  a  distribution-free  or  non-parametric  technique.  Uncertainties  exist  in  the  frequency  curves  from  either 
analytic  or  non-analytic  methods.  The  methods  used  to  quantify  those  uncertainties  vary,  however.  EC 
1 1 05-2-XXXX  describes  the  methods  to  be  used  to  quantify  uncertainty  in  analytic  frequency  curves.  This 
ETL  describes  the  non-parariietric  order  statistics  approach  for  quantifying  uncertainty. 

The  order  statistics  approach  is  based  on  a  straightforward  application  of  the  binomial  distribution. 
The  ETL  is  well  supported  in  a  technical  sense,  which  of  course  is  the  nature  of  an  ETL.  However,  it  is, 
unfortunately  not  as  reader  friendly  as  risk  and  uncertainty  guidance  needs  to  be  if  it  is  to  be  used  and 
understood  by  the  widest  audience  possible. 

There  are  limits  to  how  much  some  of  the  complex  statistical  procedures  and  concepts  can  be 
digested  for  users.  There  are  legitimate  arguments  about  the  desirability  of  even  attempting  to  do  that;  “a 
little  knowledge  being  a  dangerous  thing”  chief  among  them.  Nonetheless,  when  binomial  probabilities  can 
be  expressed  with  notation  no  more  complex  than  a  factorial,  we  question  the  wisdom  of  offering  an  equation 
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like  (2)  on  page  2  that  involves  the  incomplete  beta  function  and  integral  calculus  as  “a  more 
convenient.  ..expression”. 

The  guidance  is  still  well  prepared.  It  proceeds  from  a  mathematical  presentation  of  the  order 
statistics  concept  to  an  example  that  takes  most  if  not  all  of  the  sting  out  of  the  math  presented  earlier.  The 
guidance  provides  sufficient  discussion  of  related  issues  to  satisfy  most  situations  and  there  are  practical 
guidelines  for  determining  equivalent  record  lengths.  There  are  numerous  useful  examples. 

On  balance,  this  ETL  provides  a  good  example  of  the  type  of  guidance  that  constitutes  adequate 
support  for  an  integrated  uncertainty  model.  However,  at  this  point,  it  is  unclear  whether  or  where  a  single 
integrated  risk  model  will  be  used  for  environmental  resource  planning.  If  and  when  such  a  model  is 
developed  for  use,  this  ETL  would  serve  as  a  reasonably  good  example  of  the  type  of  risk  analysis  guidance 
that  should  accompany  it. 


SUMMARY 

Existing  Corps  guidance  for  risk  analysis  tends  to  consist  of  two  types:  1)  the  general  concepts, 
principles  and  tools  as  discussed  in  manuals  and  IWR  reports;  and  2)  detailed  and  specific  guidance  as 
presented  in  ECs  and  ETLs.  There  is  much  in  the  former  guidance  that  can  be  readily  incorporated  into 
environmental  risk  analysis  guidance.  There  is  much  in  the  latter  guidance  to  be  emulated  by  environmental 
risk  analysis  but  very  little  that  can  be  directly  adapted. 

At  this  point  when  we  speak  of  environmental  risk  guidance  we  are  not  talking  about  ECs  and  ETLs. 
We  are  speaking  more  about  introducing  this  relatively  new  concept  of  risk  analysis  to  the  evolving  subfield 
of  water  resources  and  environmental  planning.  The  most  reasonable  and  modest  goals  for  the  initial 
guidelines  would  include:  identifying  basic  theories,  techniques  and  concepts  used  in  risk  and  uncertainty 
analysis;  applying  them  to  representative  or  commonly  encountered  environmental  planning  situations; 
taking  a  first  cut  at  identifying  key  quantities/variables;  developing  some  precision  in  the  language  of 
environmental  investment  planning;  establishing  procedures  for  dealing  with  expert  disagreement  and  public 
controversy;  encouraging  model  reviews  and  development  of  new  models;  and  fostering  an  exchange  of 
ideas  among  agency,  academia  and  public  interests. 

Sample  analyses,  literature  reviews,  lists  of  key  variables,  introductions  to  the  array  of  techniques 
and  tools  available  for  risk  and  uncertainty  analyses,  and  models  to  be  emulated  in  structuring  future 
environmental  risk  analysis  guidance  are  among  the  most  useful  things  to  be  gleaned  from  the  Corps  existing 
guidance.  At  this  writing  there  were  several  additional  examples  of  risk  analysis  guidance  under  preparation. 
Upon  their  completion  they  warrant  review  as  well. 
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CHAPTER  THREE:  REVIEW  OF  PROFESSIONAL  LITERATURE 

OVERVIEW  AND  OBSERVATIONS 


INTRODUCTION 

The  main  feature  of  this  chapter  is  a  summary  of  52  books,  reports,  papers  and  articles  related  to  risk 
analysis.  It  is  by  no  means  the  only  feature  of  this  chapter.  In  this  overview  section  we  briefly  describe  the 
review  methodology.  Of  particular  interest  in  this  section  are  some  general  findings  we  have  culled  from 
the  literature  review  and  present  as  a  series  of  observations.  The  next  two  sections  contain  the  52  reviews 
presented  in  two  parts:  general  methodology  of  risk  analysis  and  applications  of  risk  analysis  to 
environmental  issues. 

The  fourth  section  is  written  for  analysts  who  will  want  to  continue  to  keep  abreast  of  developments 
in  environmental  risk  analysis  in  the  professional  literature.  It  is  not  an  exhaustive  list  of  sources.  Instead, 
it  is  a  guide  to  some  of  the  most  promising  sources  of  information  that  are  likely  to  be  of  most  interest  to 
Corps  analysts.  A  more  complete  list  of  potential  sources  can  be  obtained  from  the  52  summaries,  the 
references  included  among  the  books,  papers  and  articles  cited  in  the  works  summarized,  and  the 
bibliography  that  follows  in  the  fifth  and  final  section  of  Chapter  Three. 


LITERATURE  REVIEW  METHODOLOGY 

Risk  and  uncertainty  analysis  for  environmental  investments  must  address  issues  at  both  the 
watershed  and  project- specific  levels.  The  emphasis  of  this  literature  review  was  clearly  focused  on  the 
environmental  aspects  of  the  program  as  opposed  to  the  economic  or  engineering  aspects  of  risk  analysis. 
We  were  also  Interested  in  including  the  uncertainty  attending  biological  modeling  tools  currently  used  by 
the  Corps.  There  were  hopes,  for  example,  that  habitat  evaluation  models  like  Habitat  Evaluation  Procedure 
(HEP),  Wetlands  Evaluation  Techniques  (WET),  and  others  listed  in  EC  1105-2-206,  had  been  reviewed  and 
evaluated  in  the  literature.  Unfortunately,  that  has  not  been  the  case.  There  was  also  considerable  hope  for 
the  promise  of  articles  that  explicitly  reference  environmental  risk  analysis  in  their  titles.  As  will  be  seen 
in  the  reviews,  these  articles  generally  relate  to  the  U.S.  Environmental  Protection  Agency’s  (EPA)  risk 
analysis  paradigm.  The  EPA  paradigm  is  important,  but  does  not  provide  what  we  had  hoped  the  articles 
would  reveal. 

Despite  these  apparent  disappointments  the  review  has  revealed  a  great  deal  of  Information  of 
interest  and  value  to  Corps  planners.  Thus,  it  is  important  to  understand  how  the  52  items  reviewed  were 
selected. 
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The  outset  of  this  research  could  be  called  a  “fishing  expedition”.  Library  searches  were  begun  at 
three  libraries,  each  with  access  to  different  online  search  services.  Online  searches  of  environmental 
citations  were  also  conducted.  Some  of  these  resources  will  be  described  subsequently.  Several  authors 
prominent  in  the  fields  of  risk  analysis  or  environmental  resource  planning  were  contacted  for  their 
suggestions  about  literature  relevant  to  this  review.  In  addition,  several  days  were  spent  perusing  journals 
and  books  at  two  major  research  libraries  at  the  University  of  Maryland  (College  Park  and  Baltimore  County 
campuses)  and  Johns  Hopkins  University.  Other  public  and  college  libraries  were  investigated  as  well. 

From  this  initial  search  a  preliminary  list  of  50  books,  reports,  and  articles  was  compiled.  From 
preliminary  review  of  the  50  sources  it  became  evident  that  there  was  considerable  overlap  and  redundancy 
in  the  subject  matter  of  many  of  the  items.  The  truth  is  that  very  little  has  been  written  about  environmental 
resource  planning  as  the  Corps  conducts  it.  Most  of  the  environmental  risk  literature  actually  refers  to 
human  health  impacts,  a  key  interest  of  the  EPA.  While  some  redundancy  was  welcomed  a  decision  was 
made  to  revise  the  list  of  50  articles  as  the  literature  arrived. 


The  additions  to  the  reading  list  were  developed  from  continuing  searches  as  described  above. 
However,  the  best  leads  were  developed  from  new  resources  suggested  in  the  literature  that  was  being 
reviewed.  New  online  services  were  discovered,  new  authors  to  contact  were  identified,  and  references  to 
items  provided  fresh  new  journals  to  investigate.  Thirty-two  items  from  the  original  list  of  50  and  20  new 
items,  for  a  total  of  52,  comprise  the  reviewed  items. 


HOW  TO  USE  THIS  REVIEW 

The  items  summarized  for  this  literature  review  have  been  divided  into  two  sections.  Within  each 
section,  the  articles  are  organized  alphabetically,  by  author.  First  are  the  materials  on  general  methodology, 
numbered  G-1  through  G-16,  which  cover  the  generic  tools  and  issues  of  risk  and  uncertainty  analysis. 
Second  are  the  materials  on  applications  of  risk  and  uncertainty  analysis  to  environmental  resources  issues, 
numbered  A-1  through  A-36,  which  address  risk  and  uncertainty  analysis  in  the  explicit  arena  of 
environmental  issues. 

Many  readers  will  be  interested  in  a  more  refined  division  of  the  items  reviewed.  There  are  many 
ways  in  which  these  items  can  be  subdivided  and  none  of  them  will  prove  satisfactoiy  to  all  readers.  Thus, 
we  start  from  the  premise  that  our  system  is  flawed.  Given  its  flaws,  we  have  at  least  endeavored  to  keep 
the  system  simple. 

The  articles  are  summarized  in  Table  3-1.  The  title  of  each  item  is  offered  as  the  first  level 
delineation  of  subject  matter  (the  numbers  in  the  first  column  correspond  to  our  numbering  system  for  the 
annotations).  The  year  of  publication  is  also  listed.  Only  the  first  author  listed  is  noted  in  the  table.  The 
third  column  indicates  whether  the  item  is  a  book,  a  paper  from  a  journal,  a  government  report,  or  an  article 
from  a  collection  of  writings. 
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We  have  also  attempted  to  indicate  the  type  of  reader  who  is  likely  to  find  the  article  most  helpful 
or  interesting.  We  have  grouped  all  readers  into  one  of  three  functional  groups.  “Planners”  are  those  who 
are  interested  in  the  planning  process  and  the  broader  aspects  of  the  risk  analysis,  understanding  it,  and 
weighing  appropriate  trade-offs,  for  example.  “Analysts”  are  those  who  will  have  something  to  do  with 
conducting  the  risk  analysis.  They  may  do  the  actual  assessment  of  the  risks  or  they  may  monitor  a  contract 
in  which  a  contractor  does  the  assessment.  “Managers”  are  study  and  program  managers  who  will  direct  and 
approve  work  and  may  be  called  upon  to  manage  risks  estimated  in  the  assessment. 

At  times  we  have  distinguished  the  skill  level  of  the  reader.  Items  that  are  clearly  written  as  an 
introduction  to  a  topic  are  indicated  to  be  suitable  for  “novices”.  Items  that  are  a  bit  more  difficult  to  read 
or  that  require  a  more  technical  background  to  understand  are  designated  as  “expert”  reader  items.  In  most 
cases  the  skill  level  of  the  reader  is  not  a  factor  worth  mentioning. 

When  it  comes  to  the  matter  of  what  is  in  these  items  we  had  a  bit  more  difficulty.  They  are  already 
summaries.  In  some  cases  several  hundred  pages  of  text  are  summarized  in  two  or  three.  There  are  limits 
to  how  much  the  information  can  be  reduced.  We  have  decided  to  provide  the  reader  with  some  subjective 
information  about  the  contents  of  the  item  in  the  column  labeled  “Topics”. 

During  the  preparation  of  this  report  a  list  of  topics  addressed  in  each  item  was  made.  The  list  was 
far  too  long  to  be  useful  in  any  classification  matrix.  Hence,  we  have  arbitrarily  decided  to  list  what  we 
subjectively  believe  to  be  the  three  most  important  topics  or  ideas  addressed  in  the  item.  Generally,  we  have 
tried  to  avoid  listing  topics  that  are  obvious  from  the  title.  Our  criteria  for  determining  importance  was 
simply  to  try  to  guess  what  would  be  of  most  interest  to  the  Corps  reader.  Someone  else  could  come  up  with 
an  entirely  different  set  of  topics.  The  list  of  topics  should  not  be  considered  exhaustive  or  authoritative  in 
any  fashion.  We  have  taken  the  liberty  at  times  of  reshaping  the  author’s  arguments  in  the  language  of  the 
Corps.  These  topics,  then,  can  be  interpreted  as  the  reasons  why  you,  the  reader,  might  be  interested  in  that 
item. 


Each  item  begins  with  a  complete  citation  of  the  book,  report,  paper  or  article.  There  follows  a 
section  called  “applicability”.  In  that  section  we  do  specifically  for  each  article  what  we’re  trying  to  do 
generically  in  the  summary  table.  The  “applicability”  portion  of  the  review  is  intended  to  help  you  decide 
whether  you’re  interested  in  reading  the  entire  review.  We  have  tried  to  restrict  our  editorializing  to  this 
portion  of  the  summary.  On  occasion  it  seemed  necessary  to  make  editorial  comments  in  the  body  of  the 
review,  however. 

The  “review”  portion  of  the  summary  is  where  we  summarize  the  content  of  the  item.  Sometimes 
the  review  is  very  brief  In  other  cases  we  have  used  the  author’s  own  words  in  some  detail  to  deliver 
important  points.  The  review  style  was  modified  to  reflect  what  we  felt  was  most  important  to  convey  about 
that  item  to  the  Corps  reader.  Do  not  assume  the  review  has  successfully  conveyed  the  author’s  complete 
argument.  It  would  be  impossible  to  do  so  in  a  page  or  two.  The  sole  purpose  of  the  summary  is  to  help  you 
decide  whether  it  would  be  useful  to  you  to  obtain  a  copy  of  the  article  to  read  in  its  entirety. 
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One  last  parting  bit  of  advice  for  using  this  review  is  offered.  When  you  are  interested  in  a  topic, 
get  the  book  or  article.  Whenever  you  obtain  one  of  the  items  be  sure  to  carefully  review  the  references  the 
author  of  the  item  has  used.  The  references  are  one  of  the  most  valuable  aspects  of  some  of  these  articles. 
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OBSERVATIONS 

One  cannot  read  as  much  literature  as  this  review  required  without  coming  up  with  a  few 
observations.  One  begins  to  notice  things  time  and  again  in  the  literature.  Though  not  a  primary  or  even  an 
anticipated  part  of  this  literature  review,  we  offer  some  of  the  more  significant  of  these  observations  for  the 
consideration  of  the  Corps’  reader. 

The  first  observation  is  that  the  language  of  risk  analysis  has  many  meanings.  Titles  of  books  and 
papers,  annotations,  lists  of  keywords,  and  the  contents  of  much  that  is  written  indicate  a  striking  lack  of 
uniformity  in  the  language  of  risk  analysis.  Or  is  it  uncertainty  analysis?  Or  is  it  risk  and  uncertainty 
analysis? 

Risk  and  uncertainty  analysis,  as  the  Corps  defines  it,  is  applied  in  many  different  contexts.  It 
belongs  to  no  one  discipline  but  is  used  by  many.  As  a  result,  a  vast  and  varied  jargon  has  arisen.  The  Corps 
says  risk  analysis  comprises  the  two  steps  of  risk  assessment  and  risk  management.  EPA  says  risk 
assessment  comprises  the  two  steps  of  risk  analysis  and  risk  management.  The  specific  meanings  of  these 
terms  are  flip-flopped.  There  are  numerous  similar  examples  in  the  literature. 

Initially,  we  were  buoyed  by  the  numerous  references  to  environmental  risk  analysis.  This  is  one 
of  the  names  given  to  the  quantitative  risk  assessment  (as  defined  by  EPA)  of  human  health  effects  of 
projects  and  actions.  Human  health  or  environmental  risk  assessment  estimate  the  incidence  of  death  and 
disease  to  humans  resulting  from  exposure  to  hazardous  agents.  Ecological  risk  assessment  is  a  term  that 
is  coming  to  mean  the  estimation  of  hazardous  agent  impacts  on  non-human  elements  of  the  environment 
or  the  risk  associated  with  chemicals.  Others  in  the  literature  suggest  that  human  health  and  ecological  risk 
assessments  are  subsets  of  a  broader  field  called  environmental  risk  assessment. 

Risk  analysis  is  routinely  applied  to  many  disciplines:  human  health/environment;  ecology; 
conservation  biology;  reliability  of  project  performance  (the  most  common  context  in  which  it  is  applied  in 
the  Corps);  safety;  natural  disasters;  accidents;  insurance;  business  profit  and  loss;  and  numerous  other 
applications.  Each  discipline,  each  government  agency,  and  each  program  has  adopted  its  own  language  to 
describe  what  they  do  and  what  it  means. 

As  the  Corps  seeks  to  integrate  the  use  of  risk  and  uncertainty  analysis  into  the  evaluation  of 
environmental  resource  investment  projects  it  is  going  to  need  a  language  in  which  it  can  communicate 
clearly  and  unambiguously.  As  the  review  of  Corps  guidance  in  Chapter  Two  indicates,  there  is  already  a 
substantial  investment  in  the  development  of  language  that  would  be  foolishly  ignored.  Nonetheless,  there 
remains  a  need  to  develop  new  language  carefully. 

For  example,  what  should  the  type  of  risk  and  uncertainty  analysis  that  this  entire  review  is  about 
be  called?  To  distinguish  it  from  the  very  specific  work  of  the  EPA  related  to  human  health  effects. 
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“environmental  risk  assessment,”  would  probably  not  do.  “Environmental  risk  analysis”  is  too  close  a  term 
and  would  cause  confusion  in  the  literature.  Besides,  reference  to  risk  alone  seems  to  deny  the  importance 
of  that  larger  class  of  problems  and  issues  called  uncertainty.  Perhaps  “environmental  resource  uncertainty 
analysis”  would  be  descriptive  enough  to  succeed.  It  does  not,  unfortunately,  flow  easily  from  the  lips. 

At  some  point,  as  risk  analysis  becomes  more  widespread  and  common,  the  need  to  standardize  the 
language  may  reach  a  critical  mass  among  all  the  disciplines  that  lay  claim  to  a  piece  of  risk  analysis.  Until 
that  time,  however,  we  in  the  Corps  should  strive  to  develop  a  language  that  is  clear  and  unambiguous  as  it 
becomes  necessary  to  define  terms  and  describe  procedures  and  results. 

A  second  observation  is  that  other  federal  agencies  are  actively  involved  in  risk  analysis  research 
and  development  that  could  be  of  tremendous  interest  to  the  Corps  in  general  and  to  the  environmental 
program  in  particular.  Much  of  the  literature  reviewed  in  this  report  was  prepared  by,  for,  or  as  a  result  of 
the  programs  of  other  federal  agencies.  These  include  the  U.S.  Environmental  Protection  Agency  (EPA), 
the  Department  of  Energy  (DOE),  the  Nuclear  Regulatory  Commission  (NRC),  the  Food  and  Drug 
Administration  (FDA),  the  Occupational  Safety  and  Health  Administration  (OSHA),  and  the  Consumer 
Product  Safety  Commission  (CPSC).  The  National  Academy  of  Sciences  (NAS)  plays  a  prominent  role  in 
the  history  of  much  of  this  literature  as  well. 

There  are  obvious  differences  in  authorities,  programs,  applications,  paradigms,  and  terminology 
among  the  various  agencies.  But  there  are  important  similarities  as  well;  lack  of  data,  evolving  science  and 
theory,  the  need  to  train  employees,  the  need  to  develop  and  integrate  models,  the  need  to  communicate  with 
the  public,  the  need  to  communicate  with  other  risk  analysts,  and  the  need  to  stretch  research  budgets,  for 
example.  This  would  provide  a  rationale  for  dialogue  and  cooperation  among  the  agencies.  The  related 
nature  of  certain  missions  means  that  inevitably  analyses  of  the  Corps  and,  for  example,  EPA  will  overlap 
with  one  another.  The  agencies  thus  have  a  clear  need  to  become  aware  of  what  each  is  doing  in  the  area 
of  risk  analysis. 

A  third  observation  is  that  there  are  many  talented  people  doing  a  lot  of  good  work  in  risk  analysis. 
Though  52  items  are  summarized  here,  we  had  the  opportunity  to  review  far  more  than  that  during  the  course 
of  this  research.  One  very  clear  message  is  that  there  are  a  great  many  talented  people  doing  very  good  work 
in  the  areas  of  risk  analysis  and  environmental  resource  planning.  The  authors  of  these  reports  should  be 
considered  resources. 

The  fourth  observation  is  that  there  are  many  good,  if  narrowly  focused,  periodicals  available.  In  the 
“Literature  Sources  to  Monitor”  section  of  this  chapter  we  offer  the  names  of  28  periodicals  that  would  be 
worth  monitoring  to  stay  abreast  of  future  developments  in  the  application  of  risk  analysis  to  environmental 
resource  planning  problems.  We  expect  more  and  more  risk  analysis  applications  of  interest  to  Corps 
analysts  in  the  years  ahead.  A  one-time  literature  review  like  this  may  help  but  it  does  not  provide  that 
continual  update  that  analysts  need  in  such  a  rapidly  developing  field. 

Another  observation  is  that  risk  and  uncertainty  analysis  is  everywhere  in  the  literature.  It  may  be 
explicit  or  Implicit.  The  most  natural  place  to  begin  to  look  for  information  about  risk  and  uncertainty  in 
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the  literature  is  in  the  titles  of  the  items.  The  title  search  will  most  likely  lead  to  explicit  treatments  of  risk 
and  uncertainty  analysis,  but  a  great  deal  of  good  work  could  be  overlooked  in  the  implicit  treatments  of 
uncertainty. 

Uncertainty  is  so  pervasive  that  it  can  be  found  in  virtually  every  journal  article.  Option  values  and 
existence  values  are  concepts  developed  in  attempts  to  address  fundamental  notions  of  uncertainty  regarding 
resource  values.  Articles  that  discuss  future  conditions  of  resources  are  addressing  issues  of  uncertainty 
found  everywhere.  In  searching  for  treatments  of  risk  and  uncertainty,  it  is  important  not  to  limit  one’s  self 
to  a  title  search  that  might  overlook  implicit  treatments  of  uncertainty. 

A  final  observation  is  that  there  is  a  great  deal  of  good  literature  from  which  to  choose.  As  this 
literature  review  began,  we  had  hoped  that  we  would  find  attempts  to  incorporate  risk  and  uncertainty 
analysis  into,  for  example,  the  use  of  habitat  evaluation  models  and  evaluations  of  environmental  resource 
management  measures.  The  literature  that  referred  to  environmental  risk  analysis  contributed  to  soaring 
hopes  for  a  rich  literature  that  we  only  had  to  tap  into.  Those  hopes  were  quickly  dashed.  The  environmental 
risk  analysis  invariably  referred  to  the  EPA  paradigm  for  assessing  risks  to  human  health.  The  habitat 
models  and  management  measures  have,  thus  far,  been  overlooked  in  the  literature. 

Although  we  would  be  the  first  to  admit  that  our  search  of  the  literature  should  not  be  considered 
exhaustive,  we  feel  somewhat  confident  in  saying  there  is  little  in  the  literature  that  pertains  to  risk  analysis 
applied  to  environmental  issues  that  would  be  immediately  applicable  to  the  Corps’  program.  Nonetheless, 
there  is  much  that  is  of  interest  to  the  Corps  in  the  literature,  a  point  made  numerous  times  in  these 
observations. 

In  this  review,  the  emphasis  has  been  more  on  the  application  of  risk  analysis  to  environmental 
resource  investments  than  on  the  general  method  of  risk  analysis.  There  is  a  great  deal  of  material  available 
regarding  the  general  methods  of  risk  analysis.  However,  if  an  office  or  an  individual  were  interested  in 
starting  a  library  of  introductory  source  material  for  risk  analysis  there  are  a  few  books  that  stand  out.  All 
are  summarized  in  this  report.  Morgan  and  Henrion’s  1992  book  (reference  number  G13)  has  one  of  the  best 
overall  introductions  to  the  essence,  concepts,  and  practice  of  risk  and  uncertainty  analysis  that  exists.  The 
Clemen  (1990)  text  has  a  masterful  presentation  of  decision  theory  (reference  G5).  As  a  pure  bonus  it 
provides  excellent  sections  on  modeling  uncertainty  and  preferences  as  well,  Glockman’s  1993  collection 
of  readings  is  valuable  for  the  range  of  topics  it  covers  in  a  collection  of  seminal  readings  in  the  field 
(reference  G9).  The  Science  Advisory  Board’s  1990  report  for  EPA  (reference  G16)  provides  a  short 
overview  of  that  agency’s  risk  assessment  program.  Baird’s  text  (1989)  provides  a  good  introduction  to 
Bayesian  analysis  (reference  Gl).  This  is  not  the  first  topic  a  new  analyst  will  want  to  tackle,  but  sooner  or 
later  he  or  she  will  have  to  learn  something  about  this  topic.  The  Law  book  (1991)  is  helpful  for  users  who 
will  want  to  build  simulation  models  (reference  Gil). 
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GENERAL  METHODOLOGY  LITERATURE 


Gl:  Managerial  Decisions  Under  Uncertainty;  An  Introduction  to  the  Analysis  of  Decision- 

Making.  By  Bruce  F.  Baird.  New  York:  John  Wiley  &  Sons,  1989.  pp.  xiii,  530.  Thirteen 
chapters,  six  appendices,  solutions  to  selected  problems.  ISBN  0-471-85891-9. 

Applicability 

Though  risk  and  uncertainty  analysis  is  a  field  that  continues  to  evolve,  there  are  some  topics  with 
which  every  analyst  should  be  familiar.  Bayesian  analysis  is  one  of  these  topics.  Many  environmental 
planning  analysts  are  likely  to  be  unaware  of  the  differences  between  Bayesian  and  classical  views  of 
probability.  Bayesian  analysis  provides  a  whole  new  set  of  tools  for  approaching  problems  that  may  not  lend 
themselves  well  to  classical  analysis. 

This  book  was  selected  to  provide  an  accessible  reference  book  that  introduces  the  reader  to  a  broad 
range  of  Bayesian  probability  and  Bayesian  decision  analysis  techniques.  It  is  broad  ranging  in  content, 
sacrificing  some  detail  and  depth  for  breadth.  This  is  a  real  plus  for  the  novice  reader.  While  there  are  many 
Bayesian  statistics  texts  they  may  be  too  difficult  for  beginners.  Traditional  texts  may  offer  a  chapter  or  two 
on  Bayesian  analysis  at  best.  This  text  is  a  good  introduction  to  Bayesian  analysis  and  is  replete  with 
examples.  It  is  highly  recommended  to  anyone  seeking  an  introduction  to  Bayesian  analysis.  Its  many 
examples  are  well  worth  the  time  it  takes  to  follow  them. 

Review 


Baird's  book  proceeds  from  the  notion  that  the  era  of  successful  intuitive  decision-making  is  over. 
The  goal  of  the  book  is  to  "..improve  decision-making  skills  in  realistic  situations  and  do  it  in  reasonably 
nonmathematical  fashion.”  We  think  he  succeeds  on  both  counts.  The  primary  focus  of  the  book  is  Bayesian 
decision  theory  but  it  does  far  more  than  that. 

The  book  begins  with  some  fundamental  concepts  that  define  decision-making  processes  in  an 
effective  manner.  The  first  chapter  is  a  good  basic  introduction  to  the  nature  of  decision  theory.  The  next 
three  chapters  deal  with  probability.  First,  Baird  offers  a  treatment  of  elementary  probability  theory  that 
indeed  does  succeed  at  a  nonmathematical  level.  The  strength  of  this  chapter  is  that  it  covers  all  the 
probability  concepts  important  to  decision-making  without  getting  bogged  down  in  theory  or  prolonged 
numerical  examples.  A  priori  and  empirical  objective  probabilities  are  defined  and  illustrated.  Simple, 
conditional,  Joint  and  marginal  probabilities  are  clearly  defined.  Probability  trees  illustrate  examples  of  each 
of  these  probabilities.  Baird  glides  through  the  elements  of  probability  in  a  manner  sure  to  be  appreciated 
by  non-statistician  professionals. 
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In  the  third  and  fourth  chapters  he  discusses  random  variables  and  probability  distributions.  Baird 
offers  more  detail  and  more  examples  than  the  Morgan  and  Henrion  (1992)  text  reviewed  elsewhere 
(reference  G13).  The  detail  may  be  a  bit  much  for  some,  but  most  analysts  entering  or  reentering  the  field 
of  risk  analysis  will  find  it  just  right  or  nearly  so.  The  first  four  chapters  do  very  well  what  some  texts  take 
considerably  more  pages  to  do  less  effectively. 

In  Chapter  5  Baird  discusses  decision  theory  and  inference.  Inference  is  a  conclusion  or  judgment 
about  the  nature  of  some  uncertain  quantity,  for  example  a  point  or  interval  estimate  of  a  population  mean. 
We  cross  over  from  inference  to  decision  theory  when  actions  must  be  taken  as  a  result  of  our  judgments. 
Both  inference  and  decision  theory  can  be  classical  or  Bayesian.  Section  5.1  provides  a  masterful  overview 
of  the  differences  between  Bayesian  and  classical  approaches. 

He  offers  an  example  of  the  difference  between  the  two  schools  using  a  95%  confidence  interval 
about  a  sample  mean: 

"We  are  95  percent  confident  that  the  mean  battery  life  of  the  population  lies  within  30  and 
42  months.  This  statement  does  not  mean  that  the  chance  is  0.95  that  the  mean  life  of  all 
our  batteries  falls  within  the  interval  established  from  this  one  sample.  Instead  it  means  that 
if  we  select  many  random  samples  of  this  sample  size  and  if  we  calculate  a  confidence 
interval  for  each  of  these  samples,  then  in  about  95  percent  of  these  cases,  the  population 
mean  will  lie  within  the  interval”  (p.  137). 

The  Bayesian  finds  this  too  restrictive  and  would  prefer  to  say  the  interval  30  to  42  months  contains  the 
population  mean  with  a  probability  of  0.95. 

With  clear  examples  like  this  Baird  proceeds  to  build  a  good  intuitive  feel  for  the  major  differences 
between  the  two  schools  of  thought.  Simple  and  straightforward  discussions  of  decision  criteria  like  the 
maximax  and  minimax,  for  example,  are  just  a  bonus  in  this  chapter. 

Elementary  Bayesian  concepts  are  presented  in  Chapter  6.  Beginning  with  conditional  values,  Baird 
develops  an  effective  inventory  model  to  effectively  introduce  a  number  of  concepts  that  could  become 
useful  in  risk  analysis  of  environmental  resource  plans.  These  concepts  include  expected  value  of  perfect 
information  (EVPI),  expected  profits  under  uncertainty  (EPUC),  expected  loss  (EL) ,  critical  ratios,  and  value 
of  perfect  information  problems.  The  use  of  inventory  models  may  itself  have  some  value  for  environmental 
resource  planners  if  they  can  find  some  legitimate  parallels  between  the  problem  of  having  an  optimal 
quantity  of  goods  on  hand  under  conditions  of  uncertain  demand  and  determining  optimal  levels  of 
environmental  resources. 

The  construction  of  multilevel  decision  diagrams  is  discussed  in  Chapter  7  in  a  rather  informal  and 
easy-to-follow  way.  The  essential  elements  of  the  Clemen  (1990)  text,  reviewed  elsewhere  in  this  report 
(reference  G5),  are  effectively  presented  and  illustrated  with  examples  in  a  single  chapter. 
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The  essence  of  Bayesian  analysis  is  addressed  in  Chapter  8  on  information  and  the  revision  of 
probabilities.  An  introductory  discourse  on  the  nature  of  information  effectively  characterizes  the  "data 
overload/information  shortage"  confronting  analysts  and  decision  makers  alike.  In  a  fun  and  effective 
challenge  of  the  readers  knowledge  of  facts,  Baird  makes  some  effective  points  about  degree  of  belief,  an 
essential  Bayesian  concept.  He  offers  five  commonly  known  "facts",  like  the  mention  of  an  apple  in  the 
Biblical  story  of  Adam  and  Eve,  and  then  debunks  them  to  capture  the  readers  interest  and  attention.  This 
intuitive  approach  is  found  throughout  the  book  and  it  is  quite  effective. 

It  is  in  this  chapter  that  Bayes'  theorem  is  presented.  In  essence  it  is;  “joint  probability/simple 
probability  =  conditional  probability.”  He  follows  this  simple  definition  with  an  example  that  both 
illustrates  Bayes'  theorem  and  drives  home  the  difference  between  Bayesian  and  classical  approaches  to 
probability.  His  treatment  of  prior  probability,  likelihood  and  posterior  probability  is  effective  in  the 
example  that  shows  an  apparently  obvious  decision  that  proves  to  be  incorrect.  In  the  remainder  of  the 
chapter  priors'^  are  revised  using  different  kinds  of  distributions  and  conjugate  priors  are  discussed  in  a 
simple  manner,  features  not  found  in  most  texts  that  give  a  cursory  review  of  Bayesian  probability/decision 
theory. 


The  remainder  of  the  book  is  devoted  to  a  variety  of  topics  of  interest  to  risk  analysts.  Probability 
assessments  based  on  both  historical  data  and  personal  assessment  are  described.  Utility  theory  is  developed 
effectively,  as  are  sensitivity  analysis  and  multicriteria  decision  problems.  The  treatments  of  these  latter 
concepts  are  consistent  and  effective.  The  use  of  examples  continues  and  some  of  the  latter  chapters  are  a 
bit  more  mathematical,  but  the  math  rarely  goes  beyond  algebra.  The  many  graphs  are  helpful,  once  one  gets 
used  to  the  idiosyncrasies  of  Baird's  decision  tree  style.  Though  done  well,  some  of  these  topics  are  treated 
more  effectively  in  other  books  and  articles  reviewed  in  this  report. 

Appendix  VI  offers  a  very  valuable  probability  assessment  calibration  that  could  be  useful  to 
analysts  planning  to  become  involved  in  subjective  probability  assessments.  Calibration  is  explained  by 
Baird,  who  offers  a  series  of  75  questions  that  can  be  used  to  calibrate  the  subjective  tendencies  of  so-called 
experts  along  with  a  scoring  rule.  These  questions  reveal  whether  a  person  tends  to  be  over  confident,  over 
cautious  or  "just  right"  in  estimating  probabilities  of  uncertain  events.  This  appendix  could  come  in  handy 
for  analysts  who  eventually  begin  to  do  probability  assessments  except  for  one  surprising  and  annoying  flaw: 
the  answers  are  not  included! 


“Priors”  refer  to  prior  or  original  probabilities,  or  beliefs  about  the  value  of  a  parameter  held  “prior  to”  or 
before  additional  information  is  collected  through  sampling  or  testing.  Prior  probabilities  combined  with  information 
from  a  test  or  sample  yield  posterior  probabilities,  i.e.,  probabilities  that  incorporate  prior  beliefs  and  test  results. 


48 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography 


G2:  ‘‘The  Role  of  Environmental  Risk  Analysis  in  the  Cost-Effective  Development  and  Operation 

of  Emerging  Energy  Technologies.”  By  Nathaniel  F.  Barr.  In  Annals  of  Nuclear  Energy,  Vol. 
10,  No.  3/4, 1983.  pp.  187-194. 

Applicability 

This  article  was  selected  to  inform  Corps  personnel  about  the  risk  analysis  experience  of  another 
federal  agency.  It  recounts  the  efforts  of  the  Department  of  Energy  in  an  astoundingly  frank  assessment  of 
the  DOE  environmental  risk  program.  This  article  was  written  when  risk  analysis  was  fighting  the  battle  for 
respectability  and  acceptance.  There  may  be  lessons  to  learn  from  the  efforts  of  DOE  in  the  early  days  of 
its  risk  analysis  program.  It  bears  noting  that  this  article  was  written  in  1983  and  it  is  unclear  how  long  the 
programs  described  persisted. 

Review 


This  article  begins  with  the  intent  of  using  risk  analysis  to  guide  the  acquisition  and  application  of 
‘  environmental  data  to  reduce  uncertainty  regarding  the  potential  environmental  impacts  of  emerging  energy 
technologies.  At  the  time  of  this  article  there  was  a  great  divergence  of  views  regarding  the  potential  utility 
of  risk  analysis  in  resolving  issues  of  environmental  acceptability  of  existing,  much  less  emerging,  energy 
technologies.  DOE  was  thwarted  by  the  often  encountered  problem  of  a  lack  of  data.  This  lack  of 
information  is  considered  the  greatest  source  of  uncertainty. 

At  this  point  the  article  takes  a  turn  away  from  a  technical  analysis  of  the  risk  analysis  issues  toward 
an  assessment  of  DOE’s  risk  research  program.  This  shift  in  emphasis  provides  an  alternative  model  for 
organizing  an  agency’s  risk  research. 

doe’s  Office  of  Health  and  Environmental  Research  (OHER)  is  part  of  the  Office  of  Energy 
Research.  One  of  OHER’s  principle  functions  was  to  conduct  a  research  program  that  would  provide 
information  needed  to  reduce  uncertainties  regarding  the  potential  health  and  environmental  impacts  of 
emerging  energy  technologies.  Around  1981  OHER  initiated  a  Health  and  Environmental  Risk  Analysis 
Program  (HERAP). 

The  strategy  of  HERAP  was  to  support  analyses  that  describe  what  is  known  and  what  is  unknown 
about  potential  health  and  environmental  impacts  of  emerging  energy  technologies.  The  results  of  this 
program  were  presented  annually  in  a  technology-specific  Health  and  Environmental  Effects  Document 
(HEED).  The  principle  product  of  these  analyses  was  understood  to  be  a  description  of  the  nature,  magnitude 
and  sources  of  uncertainty  regarding  potential  health  and  ecological  impacts. 

At  the  time  of  this  article  the  Third  Annual  Contractor  Meeting  had  been  held  in  1982.  These 
contractors  had  to  date  prepared  13  technology-specific  HEEDs.  A  footnote  indicates  that  these  and  future 
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HEEDs  may  be  obtained  from  the  author  at  the  U.S.  Department  of  Energy,  ER-73  (GTN),  Washington,  D.C. 
20545. 


In  an  evaluation  of  the  HERAP  the  author  finds  the  HEEDS  deficient  in  several  areas  of  vital 
importance.  First,  they  were  incomplete  in  their  coverage  of  potential  environmental  impacts.  The  analyses 
were  found  to  lack  sophistication  and  data.  In  addition  they  are  very  site-specific  and  unique.  The  HEEDs 
had  provided  little  of  value  for  guiding  ecological  research. 

With  regard  to  health  impacts,  the  HEEDs  did  not  make  it  clear  how  individual  uncertainties 
contribute  to  overall  uncertainty.  This  was  to  have  been  addressed  by  development  and  use  of  more  uniform 
procedures  for  dealing  with  uncertainty. 

Of  potential  interest  to  the  Corps  as  it  develops  risk  analysis  for  environmental  programs  is  the 
HERAP  review  process.  HEEDs  were  reviewed  by  a  community  of  experts  with  broad  responsibilities  for 
the  management  of  environmental  issues.  At  the  time  the  article  was  written  DOE  intended  to  expand  the 
review  of  their  work  to  include  the  National  Academy  of  Sciences/National  Research  Council.  The  NRC 
was  to  undertake  an  independent  review  and  critique  of  the  HEEDs. 

Overall,  DOE  experience  indicated  that  risk  analysis  was  most  effective  as  an  integral  part  of  the 
decision  process,  rather  than  an  add-on.  Additionally,  they  found: 

“..science  as  well  as  public  policy  are  better  served  by  timely  organization  and  display  of 
current  uncertainty  to  support  decisions  than  are  served  by  delay  and  avoidance  while 
attaining  greater  certainty  regarding  potential  environmental  impacts”  (p.  193). 
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G3:  "Unacceptability  of  Acceptable  Risk."  By  A.  Bicevsids.  Search.  Vol,  13,  No.  1-2, 

February/March  1982.  pp.  31-34. 


Applicability 

How  do  we  determine  when  the  risk  created  by  an  action  or  the  risk  remaining  after  an  action  is 
acceptable  to  the  general  public?  If  zero  risk  is  not  a  viable  option,  how  much  risk  is  acceptable?  There  is 
no  simple  answer  to  this  question.  This  article  does  little  to  clarify  the  issues. 

Review 


The  author’s  view  on  acceptable  risk  is  never  in  doubt  in  this  article:  ’’..the  chimera  of  an  ’acceptable’ 
risk  has  led  to  a  series  of  subsequent  developments  in  risk  analysis  attempting  to  quantify  a  concept  which 
has  no  right  of  existence  in  the  first  place"  (p.  34).  Bicevskis  offers  a  brief  history  of  the  concept  of 
acceptable  risk  as  it  had  developed  in  the  literature  to  that  point  in  time.  This  review  is  interesting  to  those 
seeking  the  historical  development  of  the  argument  but  is  of  little  practical  interest  today. 

Many  of  the  author's  arguments  do  not  ring  true.  For  example,  the  author  argues  that  cost-effective 
allocations  of  risk-reducing  resources  do  not  even  raise  the  issue  of  absolute  or  acceptable  risk.  We  believe 
that  setting  the  dollar  limits  on  these  resources  in  fact  does  place  us  in  a  de  facto  acceptable  risk  range.  The 
acceptable  risk  becomes  what  we  can  afford  given  our  budgets.  The  fact  that  there  is  no  explicit  discussion 
of  the  matter  does  not  obviate  the  issue. 

The  author  feels  strongly  that  focusing  on  the  acceptable  level  of  risk  misdirects  public  attention  and 
priorities  from  more  important  things.  He  prefers  more  of  a  multi-attribute  decision  process.  Perhaps  the 
thrust  of  this  article  is  best  summarized  in  the  author’s  own  words:  ’’..a  decision  process  which  aims  at 
promoting  the  long-term  welfare  of  the  community  at  large  should  be  based  on  a  holistic  approach 
considering  the  combined  effect  of  all  the  factors,  whereas  the  concept  of  an  acceptable  risk  deflects  public 
concern  to  one  specific  area  which  may  militate  against  the  best  overall  solution"  (p.  34). 
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G4:  Tlip  Social  Response  to  Environmental  Risk:  Policy  Formulation  in  an  Age  of  Uncertainty. 

Edited  by  Daniel  W.  Bromley  and  Kathleen  Segerson.  Boston,  MA:  Kluwer  Academic 
Publishers,  1992.  pp.  ix,  216.  Eight  articles. 

Applicability 

This  book,  written  primarily  by  economists  and  decision  scientists,  claims  to  be  a  retrospective 
assessment  of  what  is  known  about  environmental  risk.  The  book  falls  well  short  of  this  stated  purpose.  It 
does  not  define  basic  terms,  present  the  four  step  environmental  risk  assessment  model,  or  address  many  of 
the  other  issues  central  to  environmental/health  risk.  It  does,  however,  provide  thoughtful  yet  relatively  easy 
to  read  articles  that  elaborate  on  the  fact  and  theory  of  public  response  to  risk.  The  view  taken  by  the  book 
is  a  big  picture  view  of  environmental  risk  as  a  body  of  public  policy.  This  makes  it  somewhat  unique 
among  the  titles  reviewed  here. 

Review 


"Entitlements  and  Public  Policy  in  Environmental  Risks"  by  David  Bromley  argues  that  property 
rights  in  environmental  policy  determine  which  party  in  a  dispute  receives  protection  form  the  state.  People 
expect  to  be  informed  about  some  risks,  protected  from  others,  and  compensated  for  still  others.  "Better  safe 
than  sorry"  seems  to  be  a  commonly  accepted  maxim  that,  as  the  Paustenbach  (1989)  book  reviewed 
elsewhere  (reference  G15)  indicates,  can  be  quite  costly  to  the  range  of  policy  options  considered. 
Alternative  entitlement  or  rights  structures  are  not  likely  to  appear  in  the  near  future  and  institutional  setups 
will  err  on  the  side  of  caution  where  ecological  and  health  effects  are  probabilistic. 

"Environmental  Risk  Perception  and  Valuation:  Conventional  Versus  Prospective  Reference 
Theory"  by  V.  Kerry  Smith  is  an  article  for  economists  interested  in  the  theory  of  why  risk  perceptions  and 
neoclassical  economic  models  often  do  not  match  one  another.  Smith  reviews  a  number  of  potential 
explanations  for  the  misalignment  of  perceptions  and  value  favoring  the  prospective  reference  theory  of  Kip 
Viscusi.  Smith  suggests  three  ways  that  public  policies  could  be  designed  to  account  for  the  differences 
between  perception  and  values.  The  article  is  of  limited  use  to  most  practitioners. 

"Risk  Perception  and  the  Perceived  Public"  by  Lola  L.  Lopes  addresses  some  fundamental  questions 
with  which  every  risk  manager  should  be  familiar.  How  much  do  people  know  about  risk?  This  question 
provides  the  entree  into  a  cursory  review  of  some  of  the  classical  literature  on  this  topic.  What  needs  to  be 
known  and  by  whom?  Here  the  author  dips  her  toe  into  the  vast  waters  of  risk  communications  but  she  does 
not  wade  in  very  far.  The  discussion  is  largely  anecdotal  and  cursory. 

"The  Media  and  Public  Perceptions  of  Risk:  How  Journalists  Frame  Risk  Stories"  by  Sharon 
Dunwoody  looks  at  the  role  of  the  media  in  presenting  stories  about  risk.  Studies  find  that  the  media 
coverage  of  risks  does  not  mirror  reality  and  that  risk  stories  contain  very  little  risk  information.  The  article 
offers  a  series  of  findings  related  to  this  thesis.  For  example,  events  are  emphasized  over  processes  in  story 
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selection.  Such  insights  may  be  valuable  to  some,  but  the  article  concludes  with  the  observation  that  we 
know  precious  little  about  how  journalists  construct  stories  regarding  scientific  and  technological  risk. 

"The  Policy  Response  to  Risk  and  Policy  Perceptions"  by  Kathleen  Segerson  categorizes 
environmental  risks  into  three  categories:  1)  internalized  risks  caused  by  the  person  facing  the  risk;  2) 
market-based  risks  associated  with  product  and  workplace  safety  where  there  is  a  market  relationship 
between  the  risk  generator  and  the  victim;  and,  3)  externalized  risks  where  the  risks  are  involuntary  to  the 
victim.  There  is  a  potential  role  for  government  for  each  type  of  risk  and  the  role  varies  with  the  nature  of 
the  risk.  Segerson  also  finds  a  number  of  different  approaches  for  managing  the  risks.  These  include: 
privatizing  risk  management;  ex  ante  economic  incentives  (taxes  and  deposits);  regulation;  legal  liability; 
and  victim  compensation  funds.  Significantly,  the  author  concludes  that  because  risk  management  is  likely 
to  have  multiple  goals  no  one  policy  instrument  is  likely  to  suffice. 

"Decision-Making  About  Biotechnology:  The  Costs  of  Learning  From  Error"  by  E.  J.  Woodhouse 
and  Patrick  W.  Hamlett  discusses  the  risks  posed  by  biotechnology.  They  conclude  that  there  are  more  than 
economic  and  environmental  risks.  Chief  among  these  are  the  deliberate  damage  that  could  be  done  by 
humans  using  successful  technologies;  and  the  risks  of  intentional  deployment  of  lethal  pathogens. 

"Gaining  Acceptance  for  Noxious  Facilities  With  Economic  Incentives"  by  Floward  Kunreuther  and 
Douglas  Easterling  applies  an  old  concept  to  a  new  context.  What  if  we  offered  a  benefit  package  to  the 
NIMBY  (not  in  my  backyard)  and  LULU  (locally  unwanted  land  use)  crowds  that  was  sufficient  to  overcome 
their  resistance  to  a  project?  Might  that  not  help  break  the  impasse  on  at  least  some  siting  issues?  The  basic 
problem  with  siting  noxious  facilities  is  that  the  benefits  of  such  facilities  are  perceived  to  be  general  in 
nature  while  the  costs  seem  to  fall  most  heavily  on  the  community  in  which  they  are  sited.  The  authors 
suggest  any  number  of  benefit  packages  that  might  overcome  local  resistance  provided  the  risks  of  the  site 
are  perceived  to  be  acceptably  small.  They  conclude  with  an  emphasis  on  the  need  to  mitigate  risks  to  assure 
that  an  acceptable  compensating  benefits  package  can  be  found. 

"Occupational  Safety  and  Health  in  the  1990s"  by  W.  Kip  Viscusi  offers  a  critique  of  the 
Occupational  Safety  and  Health  Administration  (OSHA) .  The  article  focuses  on  a  critique  of  the  use  of 
standards  to  control  behavior  offering  alternative  economic  incentives  like  an  injury  tax. 
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G5:  Makiinr  Hard  Decisions:  an  Introduction  to  Decision  Analysis.  By  Robert  T.  Clemen.  Boston, 

MA:  PWS-Kent  Publishing  Company,  1990.  pp.  xviii,  557.  Seventeen  chapters,  six  appendices. 
ISBN  0-534-98444-4. 

Applicability 

One  of  the  best  books  to  read  to  learn  how  to  model  decisions  is  this  one  by  Clemen.  Based  on 
anecdotal  evidence  gathered  at  risk  conferences,  Clemen’s  book  is  the  book  of  choice  for  introductory 
decision  theory  classes.  It’s  well  written,  clear,  and  full  of  good  examples.  The  book  takes  work,  however, 
because  it  is  a  rather  complete  introductory  treatment  of  the  theory.  Though  it  can  be  used  as  a  reference 
book  and  read  selectively  it  is  best  treated  as  a  text  and  read  from  start  to  finish,  at  least  from  start  to  finish 
of  the  major  sections. 

The  section  on  modeling  decisions  is  highly  recommended  to  Corps  analysts  and  planners.  Better 
than  any  one  source,  it  will  teach  the  basics  of  influence  diagrams  and  decision  trees.  These  are  going  to  be 
important  tools  in  applying  risk  analysis  to  environmental  resources.  Though  this  text  has  been  selected 
primarily  for  its  explication  of  decision  theory  it  has  extremely  useful  sections  on  modeling  uncertainty  and 
preferences.  This  book  is  a  must  for  anyone’s  basic  risk  analysis  library. 

Review 


Clemen’s  book  is  divided  into  three  sections:  modeling  decisions;  modeling  uncertainty;  and 
modeling  preferences.  The  first  section  may  be  of  most  value  to  Corps  analysts.  It  begins  with  the  elements 
of  decision  problems  and  introduces  the  reader  to  the  jargon  and  essential  concepts  of  decision  theory 
including  sequential  decisions,  uncertain  events,  outcomes,  values,  and  the  time  value  of  money.  He  then 
proceeds  to  an  extremely  well  done  discussion  of  how  to  structure  decisions  that  relies  primarily  on  influence 
diagrams  and  decision  trees.  Numerous  examples  are  presented  to  show  the  reader  how  to  use  these  tools 
to  make  choices. 

A  chapter  on  sensitivity  analysis  effectively  presents  strategies  for  approaching  sensitivity  analysis 
of  decision  problems  as  well  as  graphic  devices  (tornado  diagrams)  for  displaying  the  results  of  these 
analyses.  The  final  chapter  on  decision  modeling  is  entitled  “Creativity  and  Decision  Structuring”.  This 
interesting  chapter  offers  insightful  advice  on  how  to  think  about  alternative  actions. 

The  second  section  of  the  book  deals  with  modeling  uncertainty.  It  begins  with  the  basics  of 
probability  that  packs  a  great  deal  of  easy  to  process  information  into  a  single  chapter.  The  chapter  on 
subjective  probability  is  extremely  well  done  and  is  well  worth  the  reader’s  time.  Clemen  uses  intuitively 
appealing  examples  to  illustrate  what  often  appear  to  be  very  dense  concepts  in  other  texts.  The  chapter  uses 
probability  wheels,,  computer  outputs,  decision  trees  and  a  variety  of  functions  to  effectively  illustrate  his 
discussion. 


54 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography 


Clemen’s  chapter  on  theoretical  probability  models  is  one  of  the  best  around.  He  describes  the 
distributions  with  a  mix  of  intuition  and  rigor  that  is  very  effective.  The  case  studies  on  overbooking, 
earthquake  prediction,  and  municipal  solid  waste  are  effective  capstones  to  this  chapter.  The  chapter  on 
using  data  to  construct  probability  distributions  is  brief  but  it’s  an  effective  introduction  to  some  basic 
concepts.  The  second  half  of  the  chapter  introduces  some  Bayesian  concepts. 

Monte  Carlo  simulation  is  the  subject  of  one  chapter.  With  reference  to  the  spreadsheet 
environment,  the  author  introduces  the  basics  of  Monte  Carlo  simulation.  The  explanation  of  how  random 
numbers  can  be  generated  for  various  distributions  using  a  simple  random  number  generator  is  a  good 
introduction  to  an  important  programming  skill  that  is  taken  up  at  greater  length  in  the  review  of  the  Law 
and  Kelton  (1991)  book  (reference  G1 1). 

The  last  section  of  the  book  deals  with  preference  modeling.  The  discussion  of  risk  attitudes  will 
be  of  general  interest.  Much  of  what  follows  on  expected  utility  and  its  axioms  will  be  of  interest  only  to 
experienced  analysts  and  those  with  a  serious  interest  in  risk  analysis. 

The  chapters  on  trade-off  analysis  and  multiattribute  utility  models  are  a  bit  more  esoteric  than  most. 
They  do,  however,  present  good,  basic,  and  understandable  introductions  to  topics  that  may  well  prove  of 
considerable  interest  to  risk  analysts  dealing  with  environmental  resource  issues.  Overall,  Clemen’s  book 
is  an  invaluable  reference. 
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G6:  Summary  Report  on  Issues  in  Ecological  Risk  Assessment.  By  Eastern  Research  Group  for  the 

Risk  Assessment  Forum  of  the  U.S.  Environmental  Protection  Agency.  Washington,  D.C.:  U.S. 
EPA,  February,  1991.  Three  sections  and  eight  appendices.  EPA/625/3-91/018. 

Applicability 

In  1986  the  U.S.  Environmental  Protection  Agency  published  a  series  of  guidelines  for  carrying  out 
human  health  risk  assessments.  Recently  EPA’s  Risk  Assessment  Council  has  asked  the  Risk  Assessment 
Forum  to  organize  a  program  to  develop  ecological  risk  assessment  guidelines  for  agency-wide  use.  This 
has  been  an  on-going  process.  EPA,  with  this  report,  "..embarked  on  a  multi  year  effort  to  develop 
ecological  risk  assessment  guidelines  that  will  foster  consistency  in  the  Agency's  approach  to  evaluating  not 
only  the  risks  posed  by  conventional  stresses  such  as  toxic  chemicals,  but  also  other  anthropogenic  stresses 
such  as  habitat  loss  and  global  climate  change." 

The  summary  is  an  excellent  introduction  to  ecological  risk  assessment  issues.  There  is  also  ample 
review  of  the  human  health  risk  assessment  paradigm.  The  document  is  extremely  reader  friendly.  It  is 
highly  recommended  to  those  with  responsibility  for  conducting  or  introducing  risk  and  uncertainty  analysis 
to  environmental  resource  planning.  It  is  also  highly  recommended  to  all  analysts  seeking  to  understand  risk 
analysis  issues  related  habitat  loss,  global  climate  change  and  other  non-chemical  ecological  stresses. 

Review 


In  1983  the  National  Academy  of  Sciences  (NAS)  published  Risk  Assessment  in  the  Federal 
Government:  Managing  the  Process  (National  Research  Council,  1983).  This  report  recommended  that 
federal  agencies  establish  guidelines  for  risk  assessments.  In  1986  EPA  published  risk  assessment  guidelines 
related  to  human  health  risks.  The  U.S.  EPA  is  currently  developing  a  similar  set  of  guidelines  for  conducting 
ecological  risk  assessments. 

While  human  health  assessments  focus  on  a  single  species  (humans),  ecological  risk  assessments 
involve  multiple  endpoints  at  different  levels  of  biological  organization,  from  single  species  to  communities 
of  organisms  to  entire  ecosystems.  There  are  also  different  types  of  assessments.  Human  health  assessments 
tend  to  all  be  predictive.  Ecological  assessments  are  predictive,  retrospective  and  monitoring  studies 
concerned  with  multiple  stresses,  both  anthropogenic  and  natural,  chemical  and  non-chemical. 

Selection  of  an  appropriate  risk  assessment  paradigm  is  a  critical  focus  of  the  EPA  effort  and  the 
subject  of  the  early  sections  of  the  report.  The  human  health  paradigm  recommended  by  the  NAS,  described 
in  several  publications  reviewed  in  this  report,  comprises:  I)  hazard  identification;  ii)  dose-response 
assessment;  iii)  exposure  assessment;  and,  iv)  risk  characterization.  It  has  been  generally  agreed  that  a 
modified  version  of  the  NAS  paradigm  could  be  used  for  the  ecological  risk  assessment. 

Hazard  identification  of  the  ecological  paradigm  is  complex.  Defining  the  scope  of  the  problem, 
identifying  chemical  and  non-chemical  stresses,  the  type  of  ecosystem(s)  involved,  spatial  and  temporal 
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scaling  factors  and  identifying  the  endpoints  are  critical  steps.  Instead  of  dose-response  it  has  been  suggested 
that  stress-response  may  be  a  better  term.  It  better  conveys  the  range  of  stresses. 

For  human  health  an  exposure  assessment  refers  to  the  magnitude,  frequency  and  duration  of 
exposure  and  the  human  populations  subject  to  exposure.  In  an  ecological  context,  exposure  more  frequently 
refers  to  the  concentration  or  magnitude  of  a  contaminant  or  stress  in  the  environment.  The  recent  Relative 
Risk  Reduction  Strategies  Reports  (U.S.  ERA  SAB-EC-90-02 1  and  SAB-EC-90-021A)  clearly  identified 
nonchemical  stresses  from  human  activities  as  posing  the  highest  risks  to  the  environment.  This  finding 
moves  EPA's  interests  much  closer  to  the  Corps  as  far  as  environmental  resources  planning  is  concerned. 

The  risk  characterization  for  ecological  risk  assessments  does  not  always  have  to  be  quantitative. 
Relative  rankings  of  ecological  risks  are  considered  potentially  useful  in  this  new  area  of  endeavor. 
Interestingly,  the  report's  first  recommendation  on  the  choice  of  paradigm  is  that  terminology  be  clearly 
defined. 


In  subsequent  sections,  uncertainties  in  each  of  the  four  components  of  the  risk  assessment  paradigm 
are  described  in  enlightening  detail.  Population  modeling  is  discussed  in  Section  2.4.  This  section  describes 
matrix  projection  models,  natural  population  time-series  models,  ecotoxicity  population  models,  and  others. 
The  references  in  this  report  are  an  invaluable  treasure.  Following  section  1  is  a  comprehensive  list  of  EPA 
guidelines  found  in  the  Federal  Register.  Section  2  is  followed  by  extensive  references  to  the  risk  assessment 
literature. 

The  appendices  provide  a  mixture  of  background  papers  and  written  summaries  of  colloquium 
sessions  during  which  the  content  of  this  report  was  discussed  and  debated. 
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G7:  "Risk  Assessment  and  Management:  Framework  for  Decision-Making."  By  United  States 

Environmental  Protection  Agency,  EPA  600/9-85-002.  Washington,  D.C.,  December  1984.  pp. 
35. 

Applicability 

This  is  the  earliest  EPA  document  addressing  risk  assessment  and  management.  It  was  chosen  on 
the  basis  of  its  historical  interest  to  Corps  policy  and  risk  assessment.  Risk  analysis  has  been  a  major  thrust 
of  the  EPA,  much  more  so  than  it  has  been  for  the  Corps.  It  needs  to  be  noted,  of  course,  that  the  types  of 
risk  analysis  and  its  role  in  the  agencies'  missions  vary  markedly.  Nonetheless,  the  EPA  model  for 
addressing  risk  analysis  -  both  institutionally  and  analytically  -  deserves  review  and  scrutiny  by  the  Corps, 
to  learn  by  the  successes  and  failures  of  EPA. 

Review 


The  purpose  of  this  document  was  to  develop  the  outline  of  a  strategy  that  would  allow  EPA  to  act 
constructively,  despite  the  uncertainty  they  face,  to  improve  the  condition  of  the  environment,  to  reduce  risks 
to  human  health,  and  to  protect  and  enhance  the  quality  of  natural  ecosystems.  The  report  clarifies  EPA’s 
risk  jargon  by  distinguishing  risk  assessment  and  risk  management.  In  part  it  says: 

"Scientists  assess  a  risk  to  find  out  what  the  problems  are.  The  process  of  deciding  what  to 

do  about  the  problems  is  risk  management"  (p.  3). 

The  advantages  of  separately  identifying  the  assessment  and  management  steps  were  identified  as  follows: 
1)  risk  assessment  and  risk  management  help  set  priorities;  2)  risk  management  provides  a  context  for 
balanced  analysis  and  decision-making;  and  3)  risk  assessment  and  management  produce  more  efficient  and 
consistent  risk  reduction  policies. 

The  emphasis  of  this  early  document  rests  clearly  on  reducing  the  damaging  effects  of  pollution. 
The  risk  assessment  focus  was  on  human  health  risk  assessment.  The  discussion  makes  it  clear  that  risk 
assessment  does  not  enjoy  the  status  of  incontrovertible  scientific  agreement.  Despite  its  uncertainties, 
however,  risk  assessment  is  appreciated  for  being  the  only  tool  available  for  discriminating  among 
environrnental  health  problems.  The  risk  assessment  done  by  EPA  was  seen  as  a  literal  risk  assessment.  An 
example  is  estimating  the  probability  of  developing  cancer  as  a  direct  result  of  chemical  exposure. 

The  report  describes  population  risks  from  toxic  pollutants  as  a  function  of  two  measurable  factors: 
hazard  and  exposure.  To  cause  a  risk,  a  chemical  has  to  be  toxic  (it  presents  a  hazard)  and  present  in  the 
human  environment  at  some  significant  level  (human  exposure).  Risk  assessment  judges  whether  an  adverse 
effect  will  occur  as  a  result  of  the  hazard  and  exposure.  The  total  effects  of  the  risk  are  quantified  to  the 
extent  possible.  The  basic  approach  to  risk  assessment  is  described  in  the  document  in  four  steps,  which 
have  come  to  be  identified  as  the  environmental  risk  assessment  paradigm. 
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Hazard  identification  involves  weighing  the  available  evidence  and  deciding  whether  a  substance 
exhibits  a  particular  adverse  health  effect.  Once  a  chemical  is  identified  as  a  hazard,  its  potency  is 
determined.  In  a  dose-response  assessment  the  analyst  determines  the  response  to  various  levels  of  exposure 
(dose).  The  exposure  assessment  determines  the  likely  degree  of  human  exposure  to  the  chemical  of 
concern.  In  the  final  step,  risk  characterization,  the  risk  associated  with  the  exposures  are  estimated.  The 
final  calculations  are  straightforward,  exposure  times  potency  or  unit  risk.  However,  presentation  of  the 
assessment  is  problematic  because  of  the  copious  uncertainties  associated  with  the  assessment. 

EPA  identified  several  significant  uncertainties.  First,  most  risk  assessments  depend  on  animal  tests. 
Extrapolating  results  to  humans  leads  to  a  weight  of  evidence  problems  in  hazard  identification.  Because 
it  is  not  entirely  clear  that  safe  levels  or  thresholds  exist  for  toxic  chemicals  there  are  uncertainties  in  dose- 
response  assessments.  Uncertainty  in  exposure  assessment  stems  from  data  and  measurement  limits  in 
monitoring  exposure. 

As  the  EPA  struggled  with  the  development  of  principles  and  guidelines  for  risk  assessments  it 
confronted  issues  of  terminology  and  the  degree  to  which  procedures  can  be  appropriately  standardized.  In 
Chapter  2  of  the  document,  reference  is  made  to  numerous  EPA  documents  that  may  be  of  interest  to  those 
interested  in  the  evolution  of  the  agency's  risk  analysis  policies  and  practices  and  those  interested  in  human 
health  risk  assessments.  At  the  time  this  document  was  prepared  EPA  was  in  the  process  of  completing  or 
revising  guidelines  on  carcinogenicity,  mutagenicity,  reproductive  effects,  systemic  effects,  assessment 
methods  for  chemical  mixtures,  and  exposure  assessment.  A  Risk  Assessment  Forum  was  established  at 
this  time. 

Chapter  3  addresses  the  goals  and  applications  of  risk  management.  EPA  identifies  a  duality  of  this 
management  initiative  that  may  or  may  not  materialize  for  the  Corps  in  the  area  of  environmental  resource 
planning.  Individual  risk  management  decisions  may  be  seen  as  balancing  risk  reduction  against  resources. 
On  the  other  hand,  the  program  as  a  whole  is  designed  to  balance  risk  against  risk.  That  is,  it  is  essential  to 
address  the  worst  and  most  controllable  risks  first.  To  do  otherwise  means  we  reduce  less  risk  than  we  could. 

The  primary  elements  of  risk  management  are  setting  priorities  among  risks  and  choosing  reduction 
measures  for  the  risks  selected.  The  three  major  components  of  risk  management  that  have  to  be  balanced 
are:  1)  the  harmful  effect  of  the  pollutant;  2)  cost;  and  3)  confidence.  The  effects  factor  is  not  easy.  It  can 
be  quantified  as  a  risk  calculation  but  benefits  of  reducing  effects  are  not  easy  to  quantify.  Costs  are  the 
costs  of  reducing  the  hazard  and  they  may  be  explicit  or  implicit.  The  enormous  range  in  uncertainty  makes 
the  measure  of  confidence  a  particularly  difficult  issue. 

To  analyze  the  relationship  between  cost  and  effect  several  tools  were  identified.  These  are:  benefit- 
cost  analysis;  risk-benefit  analysis;  and,  cost-effectiveness  analysis.  The  agency's  risk  management 
initiatives  at  the  time  the  document  was  prepared  were:  building  the  information  base;  using  risk 
management  tools;  and,  strengthening  the  role  of  communication  in  risk  management.  Although  the 
document  is  basic  and  somewhat  dated,  for  its  historical  and  epistemological  lessons,  it  is  worth  reading. 
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G8:  "The  Greening  of  Risk  Assessment:  Towards  a  Participatory  Approach".  By  Frank  Fischer, 

pp.  98-115.  In  Business  and  the  Environment:  Implications  of  the  New  Environmentalism. 
Edited  by  Denis  Smith.  New  York:  St.  Martin's  Press,  1993.  ISBN  0-312-09518-X  hardcover. 

Applicability 

This  article  is  applicable  to  anyone  involved  in  project  management  or  public  participation  programs. 
Environmental  resource  risk  assessments  can  be  expected  to  be  emotional,  controversial,  and  contested 
issues  in  many  studies.  Fischer  provides  a  convincing  argument  for  a  participatory  approach  to  risk 
assessment  that  may  well  become  an  essential  element  of  environmental  resource  risk  assessments  in  the 
future.  The  article,  though  directed  to  the  Environmental  Protection  Agency's  programs,  is  a  well  referenced 
non-technical  argument  to  do  risk  assessments  differently.  EPA's  reliance  on  a  scientifically  based 
quantitative  risk  assessment  is  basically  the  same  approach  used  by  the  Corps  of  Engineers  today.  Fischer 
says  this  method  can  sometimes  make  matters  worse.  Fortunately,  he  offers  enough  of  a  hint  of  a  solution 
to  give  Corps  managers  and  analysts  a  direction  in  which  to  go. 

Review 


"The  number  of  people  who  believe  that  we  live  in  the  riskiest  of  times  has  increased 
dramatically  over  the  past  two  decades.  For  those  people,  particularly  those  of  the  green 
persuasion,  the  world  is  on  the  brink  of  an  ecological  disaster"  (p.  98). 

With  these  two  sentences  Fischer  captures  the  essence  of  what  project  managers  may  find  to  be  the  attitude 
of  many  environmental  resource  study  stakeholders.  His  arguments  have  merit,  as  the  quests  for  safety  and 
environmental  protection  have  emerged  as  paramount  political  issues. 

Fischer  recounts  how  emerging  environmental  philosophies  appeared  to  verge  on  opposing  any 
project  or  technology  that  was  not  risk  free.  The  primary  response  to  this  development  has  been  to  shift  the 
resulting  political  discourse  to  the  search  for  an  "acceptable  risk".  During  the  1980s  the  EPA  instituted  a 
scientifically  based  risk  assessment  as  the  mandated  criterion  for  risk  related  decision-making.  This  process 
was  to  provide  a  technically  rational  basis  for  centralized  regulatory  decision-making. 

Briefly,  the  process  generally  requires  complex  systems  to  be  broken  down  into  various  measurable 
components;  measuring  the  statistical  probabilities  of  failures  or  other  outcomes;  examining  crucial  variables 
that  might  cause  or  exacerbate  a  failure/outcome;  and  modeling  this  entire  process.  The  result  was  intended 
to  be  the  replacement  of  risk  "perceptions"  of  technically  uninformed  stakeholders  with  objective  statements 
of  risk.  The  method  has  largely  failed  to  reassure  the  public.  Fischer  suggests  that  this  has  been  because 
managers  and  decision  makers  have  failed  to  see  large-scale  technological  systems  as  social  phenomenon. 

Risk  assessment  in  a  democratic  society  cannot  be  based  on  the  a  priori  findings  of  experts  explained 
to  groups  of  passive  public  participants. 
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"The  first  step  towards  a  solution  is  to  be  found  in  the  theoretical  redefinition  of  large-scale 
technological  systems.  Rather  than  a  complicated  set  of  nuts  and  bolts,  we  must  recognize 
such  systems  to  be  a  special  type  of  social  phenomenon"  (p.  103). 

Post  mortems  of  recent  disasters  such  as  Bhopal,  Chernobyl,  Three  Mile  Island,  Challenger,  Exxon  Valdez, 
and  innumerable  plane  crashes  reveal  organizations  and  managerial  systems  to  be  at  fault.  This  engenders 
considerable  distrust  of  experts,  government  organizations,  and  management  systems.  The  public's  trust  in 
our  ability  to  ensure  personal  and  environmental  protection  have  been  inexorably  shaken.  "Rather  than  being 
a  technical  issue  plagued  by  social  perceptions,  the  risk  problem  turns  out  to  be  as  much  a  social  question 
related  to  technical  issues." 

What  clearly  arises  from  Fischer's  article  is  the  conclusion  that  the  solution  to  public  acceptance  of 
risky  projects  and  technologies  is  not  a  well  executed  quantitative  risk  assessment  carefully  communicated 
to  the  public.  Technical  knowledge  coupled  with  improved  communications  of  empirical  findings  cannot, 
alone,  answer  the  questions  posed  to  planners  and  decision  makers  in  the  social  and  political  world.  The 
solution,  Fischer  suggests,  is  participatory  risk  assessment. 

According  to  Fischer,  the  risk  assessment  process  must  be  opened  up  to  non-experts.  There  must 
be  a  full  discussion  of  the  social  and  institutional  issues  upon  which  the  risk  assessment  depends. 
Laypersons  need  a  role  in  this  discourse  from  the  outset  of  the  analysis.  A  wider  range  of  stakeholders  needs 
to  be  incorporated  into  the  initial  problems  of  what  the  risk  problems  are  or  are  perceived  to  be.  The 
advantages  of  this  approach  are  twofold. 

"First,  it  builds  into  the  analytical  process  the  stakeholders'  pragmatic  experiential 
knowledge  about  technical  and  institutional  risks;  and,  second,  it  addresses  the  essential 
issues  of  public  legitimation  and  motivation"  (p.  107). 

The  rewards  are  credibility  and  acceptance  for  the  risk  assessment  as  stakeholders  become  cooperative 
participants  in  the  formation  of  scientific  arguments,  rather  than  passive  listeners. 

Risk  assessment  is  a  product  of  a  bureaucratic  system.  It  is  a  tool  designed  to  guide  hierarchical 
decision-making.  As  such,  scientist/analysts  can  be  expected  to  be  reluctant  to  admit  non-experts,  for  a 
variety  of  reasons;  the  outrageous  unscientific  nature  of  such  participation  chief  among  them.  However,  po 
decision  process  that  hopes  to  succeed  in  a  democratic  society  can  long  ignore  the  voice  of  those  affected 
by  the  decisions.  Fischer  provides  a  compelling  argument  for  including  all  stakeholders  in  the  risk 
assessment  from  the  very  beginning. 
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G9:  Readings  in  Risk.  Edited  by  Theodore  S.  Glockman  and  Michael  Gough.  Washington,  D.C.: 

Resources  for  the  Future,  1993.  pp.  xiv,  262.  Nineteen  articles.  ISBN  0-915707-55-1. 

Applicability 

This  book  comprises  19  reprinted  papers  authored  by  34  experts  in  13  different  journals.  The  articles 
were  assembled  to  provide  a  basic  reader  for  a  course  in  risk  analysis.  The  articles  are  all  self-contained. 
The  topic  begins  with  the  first  sentence  and  ends  with  the  last.  The  reader  need  not  have  any  sophisticated 
technical  background  although  some  math  appears  from  time-to-time.  The  selection  of  articles  represents 
the  broad  range  of  disciplines  that  deal  with  risk,  for  example,  biostatistics,  chemistry,  toxicology, 
epidemiology,  economics,  engineering,  operations  research,  psychology,  physics,  and  communications. 

The  book  is  organized  into  six  parts:  1)  basic  concepts  (four  articles);  2)  risk  comparisons  (three 
articles);  3)  regulatory  issues  (three  articles);  4)  health  risk  assessment  (three  articles);  5)  technological  risk 
assessment  (three  articles);  and,  6)  risk  communication  (three  articles).  For  the  reader  who  is  approaching 
the  topic  of  risk  for  the  first  time  this  is  a  valuable  book.  It  presents  basic  definitions,  fundamental  concepts, 
seminal  arguments,  and  informed  discussion  with  uniform  quality.  It  is  recommended  for  the  generalist 
reader  interested  in  a  broader  background  in  the  basics  of  risk  analysis. 

Review 


The  Basic  Concepts  section  offers  articles  that  help  develop  the  terminology  and  concepts  of  basic 
risk  considerations.  "Probing  the  Question  of  Technology-Induced  Risk"  by  M.  Granger  Morgan  first  poses 
the  question,  "How  safe  is  safe  enough?"  In  defining  exposure  processes  and  effect  processes  he  defines  a 
way  to  systematically  think  about  technological  risks.  Briefly  exploring  some  of  the  rules  of  thumb  that 
people  use  to  think  about  risks,  he  concludes  that  overconfidence  in  our  own  thought  processes  may  be  a 
major  pitfall. 

In  a  follow-up  article,  "Choosing  and  Managing  Technology-Induced  Risk,"  Morgan  expounds  on 
the  distinctions  between  risk  assessment  and  risk  management.  He  offers  four  generic  categories  for 
developing  risk  abatement  strategies:  1)  modify  the  natural  or  human  environment;  2)  avoid  or  modify 
exposure  processes;  3)  avoid  or  modify  effects  processes;  4)  mitigate  or  compensate  for  effects.  Morgan 
presents  the  well  known  loss  function  modified  to  show  how  a  theoretical  optimal  level  of  risk  can  be  arrived 
at.  This  optimal  level  does,  of  course,  allow  for  the  trade-off  between  risk  losses  and  the  cost  of  risk 
abatement.  Several  tools  for  quantitative  risk  analysis  are  mentioned  and  a  useful  probabilistic  risk 
assessment  example  is  presented  in  a  sidebar. 

"Defining  Risk"  by  Fischoff  et  al.  essentially  concludes  that  there  is  no  one  definition  of  risk  that 
serves  all  purposes  adequately.  The  authors  explore  the  dimensions  of  the  risk  definition  controversy  and 
offer  examples  of  risk  indices. 
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"Risk  Analysis:  Understanding  ‘How  Safe  is  Safe  Enough?’"  by  Derby  and  Keeney  treats  one  of 
the  most  fundamental  conclusions  of  the  early  literature.  There  is  no  such  thing  as  a  risk  free  solution.  Zero 
risk  is  not  a  viable  option  for  society.  Furthermore,  there  is  no  single  answer  to  "how  safe  is  safe  enough?” 
because  there  is  no  single  problem.  "The  problem  appears  in  many  different  contexts  and  solutions  for  each 
case  are  context  and  problem  specific.” 

Putting  risks  in  perspective  requires  that  they  be  compared  with  other  risks.  In  the  second  section. 
Risk  Comparisons,  the  selection  of  articles  offers  insights  into  the  basics  of  comparing  risks.  In  Wilson's 
"Analyzing  the  Daily  Risks  of  Life,"  the  author  presents  a  humorous  view  of  some  trivial  activities  with 
equal  risks.  For  example,  eating  40  tablespoons  of  peanut  butter  is  as  risky  as  canoeing  for  six  minutes.  His 
tongue-in-cheek  point  seems  to  be  that  we  worry  so  much  about  trivial  risks  because  we  have  been  so 
successful  in  alleviating  many  of  the  bigger  ones. 

"Rating  the  Risks"  by  Slovic  et  al.  is  one  of  the  classics  in  the  literature  and  is  a  must  read  for  anyone 
who  wants  to  understand  how  we  mortals  think  about  risks.  Introducing  the  heuristics  of  availability, 
overconfidence,  and  the  desire  for  certainty  the  authors  provide  keen  insight  into  our  judgmental  biases.  The 
review  of  an  expert  panel’s  assessment  of  relative  risks  is  revealing.  The  nine  risk  characteristics  presented 
have  been  reproduced  time  and  again.  This  is  one  of  the  more  interesting  and  more  important  articles  in  the 
basic  literature. 

"Ranking  Possible  Carcinogenic  Hazards"  by  Ames  et  al.  compares  quantitative  estimates  of  the 
risks  of  cancer  from  natural  and  man-made  chemicals.  The  authors  must  rely  on  animal  tests  to  identify 
carcinogens  and  argue  there  is  no  reliable  method  for  extrapolating  high  dose  animal  tests  to  low  dose 
exposure  for  humans  or  other  species.  They  propose  that  carcinogens  be  ranked  on  the  basis  of  the  ratio  of 
human  exposure  to  each  substance  to  the  dose  that  causes  cancer  in  animals.  Interestingly,  they  conclude 
naturally  occurring  chemicals  in  foods  are  a  greater  threat  than  man-made  chemicals. 

Regulatory  Issues  are  the  subject  of  part  three.  "Risk,  Science  and  Democracy"  was  written  by 
former  EPA  chief  William  D.  Ruckleshaus.  He  points  out  the  solutions  EPA  offered  to  the  pollution 
problems  of  the  1970s,  only  to  have  those  problems  surpassed  in  complexity  by  the  toxic  substances 
appearing  in  our  waste  streams.  The  author  argues  that  it  is  essential  to  keep  risk  assessment  and  risk 
management  separated.  The  science  that  supports  risk  assessment  must  continue  to  advance.  Ruckleshaus 
concludes  that  regulation  decisions  are  best  made  at  the  local  level,  within  broad  bounds  that  are  set  at  higher 
government  levels. 

Wildavsky  argues  in  "No  Risk  is  the  Greatest  Risk  of  all"  that  government  erred  when  it  took 
responsibility  for  risk  management  away  from  individuals.  He  sees  a  successful  society  paralyzed  by  a  fear 
of  risk  that  has  been  fostered  by  government  assumption  of  the  responsibility  for  risk  management.  Risks 
cannot  be  reduced  for  everyone,  he  argues.  The  desire  to  protect  us  from  all  risk  has  reduced  us  to  the 
politics  of  prevention.  Wildavsky  would  prefer  a  society  that  can  seek  gains  as  well  as  avoid  losses  and  he 
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would  prefer  that  the  responsibility  and  authority  to  do  either  rest  with  society  and  not  the  government.  A 
no  risk  society  is  neither  desirable  nor  within  reach. 

Kelman  in  "Cost-Benefit  Analysis:  An  Ethical  Critique"  says  cost-benefit  analysis  is  simply 
systematic  thinking  about  decision-making.  Beyond  this  opening  tribute  it  would  be  difficult  to  consider 
Kelman  a  strong  proponent  of  cost-benefit  analysis.  His  basic  conclusions  about  cost-benefit  analysis  and 
its  application  to  environmental,  safety,  and  health  regulations  are:  1)  a  certain  decision  may  be  right  even 
though  its  costs  outweigh  its  benefits;  2)  there  are  good  reasons  to  oppose  monetization  of  non-market 
benefits  and  costs;  and,  3)  given  these  first  two  objections  it  is  not  wise  to  devote  many  resources  to  the 
generation  of  benefit-cost  calculations.  He  opposes  the  further  spread  of  the  cost-benefit  gospel  . 

Part  four  features  Health  Risk  Assessment.  The  basic  model  comprises  four  steps.  First,  identifying 
chemicals  that  are  health  hazards  (hazard  identification).  Second,  using  biological  or  mathematical  models 
to  estimate  the  risks  associated  with  doses  below  the  doses  that  produced  the  laboratory  effects  observed  in 
laboratory  animals  or  human  beings  (dose-response).  Third,  measuring  or  estimating  the  amount  of  these 
chemicals  to  which  people/wildlife  are  exposed  (exposure  assessment).  Fourth,  the  risk  assessor  must 
characterize  the  risks  by  specifying  the  underlying  assumptions  and  uncertainties  attending  the  quantitative 
estimates  (risk  characterization). 

In  "Application  of  Risk  Assessment  to  Food  Safety  Decision-Making"  Rodericks  and  Taylor 
highlight  the  differences  in  assessing  risks  for  carcinogens  and  non-carcinogens.  Noncarcinogens  are 
generally  assumed  to  be  threshold  phenomena.  That  is,  there  is  assumed  to  be  a  dose  below  which  no  toxic 
response  will  occur.  Carcinogens,  on  the  other  hand  are  treated  as  if  any  dose,  no  matter  how  small,  is 
potentially  harmful.  The  authors  find  that  data  for  risk  assessment  are  often  very  limited  and  assessments 
compensate  for  the  lack  of  data  by  relying  on  "worst-case"  scenarios  that  intentionally  avoid  understating 
the  risks.  Not  said  is  that  there  is  an  unintentional  result  of  frequently  overstating  the  risk. 

Hattis  and  Kennedy  in  "Assessing  Risks  From  Health  Hazards:  An  Imperfect  Science"  argue  that 
despite  EPA's  efforts  to  rely  on  an  authoritative,  scientific  risk  assessment,  there  is  no  such  thing.  Scientific 
uncertainties  remain,  perhaps  even  abound.  There  are  considerable  difficulties  in  linking  cause  and  effect 
in  an  unassailable  manner.  Dose-response  relationships  are  highly  suspect  in  some  cases.  Using  animals 
to  develop  such  relationships  presents  a  whole  slew  of  extrapolation  problems.  The  authors  conclude  there 
is  a  need  for  risk  assessors  to  do  a  better  job  of  identifying,  assessing,  and  communicating  the  uncertainties 
to  policy  makers. 

In  "A  Quantitative  Estimate  of  Leukemia  Mortality  Associated  With  Occupational  Exposure  to 
Benzene"  authors  White,  Infante  and  Chu  present  a  model  linking  leukemia  to  benzene  exposure.  This 
analysis  was  responsible  for  OSHA  reversing  a  policy  on  benzene  exposure  of  workers. 

Part  five  addresses  Technological  Risk  Assessment.  Assessing  the  risks  of  large  and  complex  modem 
technological  systems  can  be  a  formidable  task,  especially  when  the  technology  is  new.  In  "Social  Benefits 
Versus  Technological  Risks"  Starr  argues  for  an  economic  decision  criteria,  "maximum  benefit  at  minimum 
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cost".  From  a  series  of  examples,  he  is  able  to  derive  five  "exploratory"  rules  of  thumb  about  how  the  public 
weighs  risk  benefits  and  costs. 

"The  Application  of  Probabilistic  Risk  Assessment  Techniques  to  Energy  Technologies"  by 
Rasmussen  provides  an  example  of  how  probabilistic  risk  assessment  techniques  are  applied.  Faults  trees 
and  event  trees  are  introduced  quite  effectively.  The  simple  calculations  reveal  the  complexity  that  can 
underlie  even  seemingly  straightforward  risk  assessments.  This  article  provides  a  good  introduction  to  some 
useful  tools. 

Keeney  et  al.  in  "Assessing  the  Risk  of  an  LNG  Terminal"  provide  a  very  readable  example  of  a  risk 
assessment.  Examining  the  shipping  of  liquid  natural  gas  (LNG)  they  conclude  that  release  of  a  vapor  cloud 
could  be  catastrophic.  However,  the  analysis  shows  that  despite  the  nature  of  this  low  probability  high 
consequence  event,  LNG  is  still  less  risky  than  conventional  means  of  generating  electricity. 

The  last  part  deals  with  Risk  Communication.  In  "The  Emergence  of  Risk  Communication  Studies: 
Social  and  Political  Context,"  Plough  and  Krimsky  indicate  that  both  the  federal  government  and  citizens 
use  or  abuse  risk  research  for  their  own  ends.  Government  officials  too  often  draw  selectively  from  the 
research  to  support  their  points  while  citizens  frequently  brush  the  research  aside  in  favor  of  more  politically 
expedient  means  to  their  ends.  Of  even  greater  importance  to  the  breakdown  in  communications  is  the 
disconnect  between  the  "technical  rationality"  of  risk  assessors  and  the  "cultural  rationality"  of  the  citizenry. 
The  authors  suggest  that  the  experts  must  recognize  the  validity  of  citizens  concerns  if  risk  communication 
is  ever  to  be  a  two-way  process. 

"Getting  to  Maybe:  Some  Communications  Aspects  of  Siting  Hazardous  Waste  Facilities"  by 
Sandman  focuses  on  the  conflicts  encountered  among  three  parties  involved  in  siting  hazardous  waste 
facilities.  Knowing  the  community  can  halt  the  siting  process  by  saying  "no"  through  legal  or  political 
means,  proponents  of  the  site  would  be  wise  to  acknowledge  the  community's  power.  Such  a  concession 
would  enable  the  community  to  come  out  from  the  comer  it  feels  backed  into  by  the  experts  and  enable  them 
to  say  "maybe". 

Johnson  et  al.  report  on  the  effectiveness  of  five  different  booklets  and  a  one  page  fact  sheet  used 
to  inform  people  about  the  risks  of  radon  in  the  home  in  "Informed  Choice  or  Regulated  Risk?".  Various 
ways  of  presenting  the  same  information  have  decidedly  different  effects  on  learning,  risk  perception,  and 
mitigation  behavior.  The  extent  to  which  people  learned  about  risk,  changed  their  risk  perceptions,  and 
formulated  advice  for  their  neighbors  about  radon  were  probed  to  establish  these  preliminary  research  results. 
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GIO  “Graphical  Communication  of  Uncertain  Quantities  to  Nontechnical  People.”  By  Harald 
Ibrekk  and  M.  Granger  Morgan.  Risk  Analysis;  An  Official  Publication  of  the  Society  for 
Risk  Analysis.  Vol.  7.  No.  4. 1987.  pp.  519-529. 

Applicability 

Effectively  communicating  the  results  of  complex  risk  analysis  to  decision  makers  and  the  public 
is  going  to  be  a  major  concern  of  Corps  analysts.  This  article  presents  the  results  of  an  experiment  in  which 
two  groups  of  people  were  provided  information  in  a  variety  of  formats.  One  group  was  composed  of 
nontechnical  people,  the  other  was  a  group  of  semitechnical  people.  The  authors  describe  their  experimental 
design  in  sufficient  detail  for  the  reader  to  appreciate  the  task  and  the  subsequently  reported  results. 

This  article  can  provide  an  education  to  Corps  analysts  in  that  it  may  suggest  some  communication 
measures  of  which  they  are  unaware.  It  is  also  useful  to  those  already  familiar  with  these  graphical 
techniques  because  it  provides  some  preliminary  evidence  of  what  works  best  before  and  after  people  receive 
some  simple  explanations. 

The  reader  is  well  advised  to  keep  in  mind  that  this  1987  article  is  somewhat  dated  in  that  statistical 
and  presentation  graphics  computer  software  has  made  these  graphics  and  many  more  very  easy  to 
reproduce.  Nonetheless,  the  article  is  clear,  concise,  accessible  and  practical. 

Review 


Without  effective  risk  communication  the  best  analysis  is  worthless.  This  paper  examines  the 
question,  “What  is  the  best  way  to  communicate,  in  graphical  form,  quantitative  information  about  the  value 
of  an  uncertain  quantity  to  semitechnical  and  nontechnical  people?”  It  additionally  considers  how  accurately 
the  information  can  be  communicated. 

An  empirical  evaluation  of  nine  graphical  techniques  is  presented.  In  the  first  test,  45  volunteers 
were  provided  a  scenario  in  which  they  were  going  to  a  mountain  cabin  and  were  concerned  about  the 
possibility  of  snow.  They  were  presented  information  in  nine  “pictures”  that  included:  an  error  bar,  a 
histogram,  a  pie  chart,  a  probability  density  function,  the  mirror  image  of  a  density  function  at  half  its 
height,  dotted  horizontal  bars,  horizontal  bars  with  vertical  lines,  a  tukey  box  (box  and  whisker),  and  a 
cumulative  distribution  function.  Without  any  explanation  of  the  graphs  they  were  asked  the  forecaster  s 
best  estimate  of  the  chances  there  would  be  more  than  2  inches  of  snow,  and  the  chances  there  would  be  2 
to  12  inches  of  snow.  In  the  second  half  of  the  test  the  people  were  given  an  explanation  of  the  graphs  and 
then  were  asked  to  make  some  predictions  about  water  depth  in  a  flood  from  a  new  set  of  graphs. 

Results  were  evaluated  in  a  number  of  ways.  Estimates  of  best  estimates,  i.e.,  means,  were  most 
accurate  with  error  bars  and  tukey  boxes.  Explanations  of  the  graphs  seemed  to  have  a  weak  effect.  For  the 
question  about  the  probability  of  obtaining  a  result  greater  than  some  value  (e.g.,  snow  greater  than  2  inches) 
the  pie  chart  and  cumulative  distribution  function  (CDF)  gave  the  most  accurate  answers.  The  CDF  also 
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gave  the  best  answers  for  the  question  of  the  probability  of  obtaining  a  result  between  two  values  (e.g.,  2  to 
12  inches  of  snow).  The  authors  offer  a  thorough  analysis  of  the  accuracy  and  confidence  people  had  in  the 
use  of  these  different  graphics.  Among  the  interesting  findings  was  that  “..a  “rusty  knowledge”  of  statistics, 
a  graduate  degree,  or  use  of  information  about  uncertainty  on  the  job,  do  not  significantly  improve  subject 
performance.”  This  does  not  bode  well  for  analysts  who  must  communicate  with  these  and  “less  well 
trained”  people. 

The  authors  offer  several  useful  conclusions,  the  first  of  which  is  that  more  study  is  needed  in  this 
area.  The  performance  of  a  display  depends  on  the  information  you’re  trying  to  convey.  If  means  are 
important  you  might  use  one  display.  For  probability  intervals  you  would  use  another.  There  are  also  some 
handy  insights  offered.  For  example,  when  asked  to  identify  a  best  estimate  people  tend  to  choose  the  mode 
rather  than  the  mean.  This  leads  us  to  the  simple  solution  of  marking  the  mean  on  a  graphic  when  it  is  the 
value  of  interest.  To  convey  probability  intervals  the  authors  recommend  using  a  CDF  printed  directly  above 
its  probability  density  function.  Until  more  work  is  done,  and  probably  well  beyond  that,  the  best  strategy 
seems  to  be  to  combine  displays  to  obtain  the  best  features  of  each. 
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Gil;  Simulation  Modeling  and  Analysis.  By  Averill  M.  Law  and  W.  David  Kelton.  New  York,  NY: 
McGraw-Hill,  Inc.,  1991.  pp.  xxii,  759.  Thirteen  chapters,  one  appendix.  ISBN  0-07-036698-5. 

Applicability 

Simulation  modeling  has  been  one  of  the  principal  tools  of  risk  analysis  as  practiced  by  the  Corps 
of  Engineers.  It  seems  this  review  would  not  be  complete  without  at  least  one  book  devoted  to  simulation. 
The  Law  and  Kelton  text  is  a  good  one.  A  helpful  feature  at  the  beginning  of  each  chapter  suggests  what  you 
should  read  the  first  time  you  read  this  book.  You’ll  have  to  read  this  book  carefully  and  more  than  once. 
This  is  a  book  for  people  who  are  getting  ready  to  start  or  who  are  already  doing  some  simulation  modeling. 
It’s  not  for  people  looking  for  information  about  simulation  modeling.  It’s  a  rigorous  text,  full  of  detail,  yet 
with  amazingly  little  of  the  complex  mathematics  that  confounds  so  many  readers.  This  is  a  text  for 
modelers  and  programmers.  For  them,  it’s  a  great  text  and  is  highly  recommended.  All  others  should  look 
elsewhere. 

Review 


This  is  a  no-nonsense  book  that  should  be  the  companion  of  every  simulation  modeler.  The  analyst 
using  a  simple  spreadsheet  simulation  won’t  need  it,  but  if  integrated  risk  analysis  models  are  going  to  be 
developed,  or  if  simulations  are  going  to  be  written  in  a  “C  language”  or  visual  basic,  this  is  the  book  you 
want. 


The  first  and  second  chapters  on  basic  and  complex  system  modeling  will  exhaust  the  reader.  They 
are  complete  treatments  with  ample  examples  and  sample  programs.  The  two  chapters  consume  234  pages. 
The  writing  is  good,  the  graphics  copious,  but  the  detail  is  draining.  It  takes  careful  reading  to  follow  the 
authors  extremely  helpful  detailed  discussions  of  subroutines.  Reading  about  sample  programs  takes  an 
uncommon  effort  to  focus  on  the  context  of  the  problem  being  discussed.  The  effort  is,  however,  well 
rewarded. 

Chapter  3  on  simulation  software  provides  a  helpful  overview  of  simulation  languages  and  general 
purpose  languages.  It  presents  discussions  of  GPSS/H,  SIMAN/Cinema,  SIMSCRIPT 115,  and  SLAMII . 
Table  3.2  provides  a  useful  comparison  of  these  simulation  languages  not  found  in  other  such  texts. 

Following  the  overview  presentations  of  the  first  three  chapters  the  authors  take  a  few  steps  back 
and  begin  to  fill  in  the  details.  There  is  a  basic  review  of  probability  and  statistics  that  is  nothing  special 
but  it  does  offer  a  few  nuggets  of  useful  information,  for  example,  why  one  should  not  replace  a  distribution 
by  its  mean. 

The  primary  reason  this  book  was  selected  for  this  review  is  because  of  the  material  in  Chapters  5 
through  8.  This  material  is  not  always  easy  but  it  provides  the  best  and  most  complete  discussion  of  random 
numbers  for  programmers  and  modelers  to  be  found  anywhere. 
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Chapter  8  is  worth  the  price  of  the  book.  Using  a  simple  uniform  random  number  generator  over 
the  interval  0  to  1,  the  authors  show  how  to  generate  random  numbers  for  twelve  continuous  distributions, 
seven  discrete  distributions,  three  correlated  distributions,  and  three  arrival  processes.  This  is  very  useful 
information  for  people  who  are  unable  to  rely  on  distribution  functions  provided  by  software  packages  like 
@RISK  and  Crystal  Ball. 

Chapter  9  addresses  model  outputs.  It  presents  numerous  display  methods  and  a  complete 
discussion  of  topics  like  transient  and  steady-state  behavior  and  measures  of  performance.  The  remainder 
of  the  text  becomes  considerably  more  complex.  It  describes  methods  for  comparing  alternative  system 
configurations,  a  technique  that  may  ultimately  be  of  considerable  interest  in  the  development  of  integrated 
risk  analysis  models  for  the  Corps.  Variance  reduction  techniques,  experimental  design  and  optimization 
methods  are  also  presented. 

Law  and  Kelton  offer  a  text  that  should  meet  many  of  the  basic  needs  of  Corps  simulation  modelers. 
It  is  complete  and  documented  with  a  great  variety  of  references.  If  you  can’t  find  what  you  want  in  the  text, 
it  can  almost  certainly  be  found  in  one  of  the  references. 
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G12:  An  Introduction  to  Risk  Analysis.  Second  edition.  By  Robert  E.  Megill.  Tulsa,  OK:  Penn  Well 
Publishing  Co.,  1984.  pp.  xiv,  274.  Nineteen  chapters  and  three  appendices.  ISBN  0-87814-257- 
6. 

Applicability 

This  is  an  introductoiy  level  book  on  risk  written  with  an  oil  well  exploration  orientation  that  does 
not  limit  its  applicability  to  environmental  planners.  The  material  is  presented  at  such  a  basic  level  that  it 
is  easily  accessible  by  any  analyst.  It  is  chock  full  of  rules  of  thumb  that  many  will  find  helpful.  The  oil  well 
orientation  may  prove  interesting  to  analysts  who  are  comfortable  with  reasoning  by  analogy,  for  there  are 
many  analogies  to  environmental  planning  issues  that  can  be  seen  throughout  the  text.  This  is  a  good 
supplemental  text  for  those  who  want  to  understand  some  of  the  probabilistic  nature  of  risk  better.  It  is 
recommended. 

Review 


The  goals  of  this  book  are,  in  the  author’s  own  words,  "...(an)  effort  to  make  simple  a  few  of  the  basic 
fundamentals  in  the  analysis  of  risk".  At  this,  the  author  succeeds.  Megill  does  an  admirable  job  of 
providing  a  truly  introductoiy  and  frequently  intuitively  appealing  explanation  to  several  important  methods 
or  concepts  in  risk  analysis.  Perhaps  the  only  criticism  of  the  book  is  that  he  did  not  go  further  in  either 
breadth  or  depth. 

In  a  workman-like  fashion,  the  author  introduces  one  basic  idea  at  a  time.  He  begins  with  the 
histogram  in  Chapter  1.  In  Chapter  2  he  introduces  the  binomial  and  normal  distributions.  The  chapters  are 
illustrated  with  effective,  if  somewhat  dated  looking,  graphics.  One  of  the  strengths  of  this  book  is  that  the 
references  include  many  of  the  classic  articles  in  the  literature.  A  weakness  is  the  flip  side:  there  are  far  too 
few  modem  references  to  the  material  treated. 

The  author  does  a  superb  job  of  introducing  the  reader  to  fundamental  eoncepts.  If  you  don't 
remember  what  a  measure  of  central  tendency  is  or  if  you've  forgotten  what  a  variance  is,  Megill  is  your 
author.  He  provides  a  simple  and  straightforward  explanation  devoid  of  the  statistical  formulas  and 
examples  that  come  with  the  modem  statistics  textbooks  that  treat  much  of  the  same  material  found  in 
Megill's  early  chapters. 

One  of  the  many  useful  things  about  this  book  is  its  frequent  reference  to  handy  little  rules  of  thumb, 
such  as  that  the  mean  deviation  of  a  normal  distribution  is  about  80  percent  of  the  standard  deviation.  You 
won't  find  these  handy  little  "factoids"  in  many  texts  anymore.  Part  of  the  book's  genius  is  its  simplicity. 
For  example,  when  explaining  the  lognormal  distribution,  Megill  offers  the  beautifully  simple  definition  as 
"Lots  of  little  values  and  not  many  big  ones".  In  the  context  of  his  example,  everything  just  falls  into  place. 
In  this  chapter  as  in  many  others  he  relates  some  aspect  of  the  chapter's  main  subject  matter  to  oil  and/or  gas 
fields. 
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Another  pleasant  surprise  and  useful  feature  of  this  book  is  the  author’s  attention  to  the  little 
historical  details.  He  points  out  things  like  how  the  ancient  Greeks  were  the  first  to  give  us  abstract  concepts 
using  numbers.  They  were  the  first  to  add  5  +  3  to  get  8  without  worrying  about  what  physical  things  they 
were  adding.  Does  material  like  this  help  you  learn  how  to  do  risk  analysis?  Not  in  the  least,  but  it  makes 
for  an  eminently  readable  book  and  it  provides  a  real  alternative  to  readers  seeking  an  interesting  review  of 
probability  and  statistics. 

Chapter  5  on  "The  Shape  of  Distributions"  may  be  the  best  in  the  book.  If  you  never  understood  the 
difference  between  a  pdf  (probability  distribution  function)  and  a  cdf  (cumulative  distribution  function),  you 
must  read  this  chapter.  Using  plain  talk  and  effective  graphics  the  author  will  have  you  understanding  more 
about  these  concepts  in  1 2  pages  than  you  did  in  a  whole  semester.  Considerable  space  is  devoted  to  the 
binomial  and  triangular  distributions.  Here  you  may  get  more  detail  than  you  wanted  from  an  introductory 
course,  but  you  will  understand  these  distributions. 

In  a  style  reminiscent  of  a  favorite  rambling  professor  Megill  jumps  back  and  forth  among  topics 
somehow  without  annoying  the  reader  because  there  is  always  a  subtle  and  ultimately  convincing  logic  to 
the  way  the  book  is  arranged,  another  of  its  unique  strengths. 

Several  chapters  are  devoted  to  opinion  analysis  and  subjective  probability.  Subjective  probability 
involves  decisions  where  very  little  information  is  available,  he  says  with  characteristic  simplicity.  The 
example  on  subjective  probability  provided  in  Chapter  12  leaves  us  woefully  short  of  something  that 
constitutes  a  really  useful  example  for  environmental  resource  planning.  But  is  it  fair  to  expect  more  from 
a  book  that  says  it  is  an  introductory  text? 

Parallels  with  environmental  problems  abound  in  the  examples  and  case  studies.  For  example. 
Chapter  12  presents  a  situation  that  requires  the  analyst  to  place  a  value  on  four  different  tracts  of  land  that 
little  is  known  about.  We  know  they  are  similar  to  other  tracts  of  land  that  have  produced  gas  and  oil  and 
that  only  a  few  geologists  know  anything  more  about  them.  The  land  may  have  alternative  uses.  With  only 
a  little  imagination  the  gas  and  oil  become  habitat  values  the  geologists  become  environmentalists  and  we 
can  pretty  much  follow  this  example  to  its  conclusion  with  some  insight  into  how  to  pursue  subjective 
probabilities  for  a  restoration  or  mitigation  project,  for  example. 

By  the  time  this  19  chapter  book  hits  chapter  14  it  is  pretty  heavily  involved  with  oil  and  gas  field 
examples  that  might  cause  the  novice  to  nod  off  or  see  no  relevance.  Even  then  there  are  nuggets  for  those 
who  persist.  Looking  for  insight,  rules  of  thumb  and  shortcuts?  If  you're  not  concerned  about  depth  of 
overall  understanding  and  the  details  of  design,  this  is  your  book. 
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G13:  TTnfPirtflintv:  A  Guide  to  Dealing  with  Uncertainty  in  Quantitative  Risk  and  Policy  Analysis. 
By  M.  Granger  Morgan  and  Max  Henrion  with  a  chapter  by  Mitchell  Small.  New  York  and 
Cambridge:  Cambridge  University  Press,  1992.  pp.  xii,  333.  Twelve  chapters.  ISBN  0-521- 
36542-2. 

Applicability 

This  is  a  must  read  book  for  anyone  who  is  trying  to  understand  the  reasons  for  and  nature  of  what 
has  come  to  be  called  risk  and  uncertainty  analysis.  The  book  is,  for  the  most  part,  an  easy-to-read  and  well 
organized  guide  to  uncertainty  analysis,  just  as  its  title  claims.  It  is  one  of  the  most  thorough  and  best 
introductory  texts  on  the  topic.  It  is  a  book  that  will  prove  as  useful  to  lower,  middle  and  top  level  managers 
charged  with  the  responsibility  of  managing  risk  and  uncertainty  as  it  will  to  the  analysts  charged  with 
assessing  it.  It  is  highly  recommended. 

Review 


Does  uncertainty  really  matter?  That  is  the  intriguing  question  the  authors  ask  in  the  first  chapter. 
They  find  uncertainty  rather  accepted  in  the  natural  sciences  and  greatly  overlooked  in  policy  research  and 
analysis.  This  makes  the  book  particularly  useful  to  an  environmental  risk  analysis  program  that  involves 
so  many  of  the  natural  sciences  and  that  will  be  charged  with  making  policy. 

The  first  two  chapters  are  used  to  motivate  the  reader  of  the  book  by  showing  that  risk  and  policy 
analyses  are  timely  and  of  major  significance  through  a  mix  of  Socratic  argument  and  case  study  examples. 
The  strength  of  the  book,  however,  lies  in  the  eight  chapters  that  follow. 

Many  analysts  and  managers  when  attempting  something  for  the  first  time  might  like  to  see  an 
example  of  that  thing  done  well.  Chapters  3  and  4  provide  as  good  an  overview  of  the  generic  issues 
attending  risk  analysis  as  will  be  found  in  the  literature  today.  The  chapters  are  well-written  but  require 
careful  and  thoughtful  reading  in  order  for  the  reader  to  catch  on  to  the  thread  of  the  authors’  logic.  Once 
grasped,  that  logic  provides  an  orderly  approach  to  virtually  any  risk  analysis  problem. 

Chapter  3  addresses  the  broad  issue  of  what  constitutes  good  policy  analysis.  Using  elements  of  the 
scientific  method  -  empirical  testing;  documentation  and  reproducibility  of  results;  explicit  reporting  of 
uncertainty;  peer  review;  and  open  debate  about  alternative  theories  -  the  authors  develop  arguments  for 
applying  the  same  standards  to  policy  analysis.  Decision  criteria  including  utility-based,  rights-based, 
technology-based  and  hybrid  criteria  are  discussed  as  potential  philosophical  frameworks  for  the  analysis. 
The  "ten  commandments"  of  good  policy  analysis  are  worthwhile  and  should  be  considered  in  any  risk 
analysis  undertaken  by  any  agency  involved  in  public  policy  decisions.  Noting  the  most  frequent  approach 
to  uncertainty  has  historically  been  to  ignore  it,  the  authors  conclude  with  examples  of  when  it  may  be  very 
important  to  consider  uncertainty. 
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Chapter  4,  "The  Nature  and  Sources  of  Uncertainty"  is  one  of  the  best  in  the  book.  Getting  a  handle 
on  this  uncertainty  business  is  often  one  of  the  most  difficult  tasks  for  analysts  and  managers  newly 
approaching  the  topic.  If  they  can  only  find  a  way  to  begin  to  understand  what  it's  all  about,  perhaps  it 
wouldn't  be  so  difficult  to  understand  after  all.  Morgan  et  al.  begin  with  a  straightforward  and  simple 
explanation  of  the  nature  of  probability.  Though  many  texts  identify  the  classical,  frequentist  and  subjective 
views  of  probability  as  three  distinct  ways  to  look  at  probability,  the  authors  give  little  weight  to  the  classical 
view  which  they  have  blended  in  with  the  frequentist  view  of  probability.  The  subjective  view  is  called 
personalist  or  Bayesian  by  the  authors.  Nonetheless,  the  treatment  is  clear,  concise  and  easily  understandable. 

The  real  value  of  this  chapter  is  found  in  the  taxonomy  of  uncertainty  that  follows  the  discussion  of 
probability.  First,  the  authors  identify  the  types  of  quantities  that  are  encountered  in  policy  models.  They 
include:  empirical  parameter  or  chance  variable;  defined  constant;  decision  variable;  value  parameter;  index 
variable;  model  domain  parameter;  and  outcome  criterion.  Next,  the  sources  of  uncertainty  encountered  in 
these  different  empirical  quantities  are  identified.  These  sources  are:  statistical  variation;  subjective 
judgment;  linguistic  imprecision;  variability;  inherent  randomness;  disagreement;  and  approximation.  If  the 
types  of  significant  quantities  encountered  in  an  analysis  are  identified  and  the  sources  of  uncertainty 
attending  them  can  be  specified,  the  techniques  for  addressing  these  uncertainties  can  be  more  readily 
addressed. 

The  chapter  concludes  with  a  brief  discussion  of  model  uncertainty,  perhaps  the  most  important  and 
overlooked  source  of  uncertainty  in  many  instances.  Model  uncertainty  describes  the  circumstances  where 
we  are  unsure  about  the  structure  or  form  of  the  model  that  best  represents  the  system  or  problem  under 
analysis.  For  example,  the  uncertain  quantities  and  sources  of  uncertainty  attending  the  use  of  habitat 
evaluation  models  can  be  identified  and  analyzed.  The  theory  and  structure  of  the  habitat  model  itself  could 
be  a  source  of  greater  uncertainty  than  all  the  uncertain  quantities  combined. 

Following  the  development  of  these  basic  concepts  the  authors  offer  an  extremely  useful  discussion 
of  probability  distributions.  Eleven  common  and  useful  families  of  distributions  are  described.  These 
descriptions  provide  a  handy  introduction  to  situations  in  which  each  distribution  may  be  appropriately  used. 

It  is  inevitable  that  environmental  resource  risk  analysis  will  confront  many  situations  where 
complex  systems  are  poorly  understood,  theory  is  not  well  defined  or  data  are  inadequate.  In  these  situations 
professional  judgment  will  become  a  most  important  method  for  ascertaining  potential  effects.  Subjective 
estimates  of  uncertain  situations  or  opinion  analysis  will  have  a  major  role  in  these  risk  analyses.  In  Chapter 
6  the  authors  provide  an  overview  of  the  major  issues  encountered  in  subjective  estimates  of  uncertainty. 
The  chapter  provides  an  excellent  review  of  the  range  of  issues  addressed  in  the  professional  literature. 

The  heuristics  of  availability,  representativeness,  and  anchoring  and  adjusting  are  illustrated.  Of 
particular  value  is  the  treatment  of  the  psychology,  elicitation,  and  evaluation  of  subjective  probability 
assessments.  This  chapter  is  a  "must  read"  for  analysts  who  will  be  conducting  risk  analyses.  The  literature 
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cited  throughout  the  chapter  provides  an  invaluable  base  for  further  exploring  this  topic  that  will  prove  most 
useful  to  environmental  resource  risk  analysis. 

Chapter  7  presents  several  examples  of  subjective  probability  elicitations.  Three  different 
probability  assessment  protocols  are  presented  in  such  a  clear  and  direct  fashion  that  the  reader  will  feel  well 
grounded  in  the  basics  of  these  techniques  after  reading  only  this  chapter.  There  is  a  wealth  of  helpful 
guidance  about  what  constitutes  a  good  protocol,  what  to  do  when  experts  won't  cooperate  and  how  to  handle 
differences  of  opinion.  In  short,  this  chapter  is  the  perfect  primer  for  conducting  probability  elicitations. 

The  most  mathematically  rigorous  section  of  the  book  is  the  discussion  of  analytical  and 
computational  techniques  for  examining  uncertainty.  Chapter  8  introduces  tools  like  decision  trees  and 
Monte  Carlo  simulation  using  a  modest  amount  of  calculus.  This  chapter  will  be  too  abbreviated  and 
mathematically  esoteric  for  beginners  and  rather  too  superficial  for  experienced  analysts. 

How  does  one  convey  the  results  of  a  risk  analysis  to  those  who  are  not  risk  analysts?  That  is  the 
question  explored  in  the  ninth  chapter.  The  results  of  numerous  studies  are  reviewed.  Probability  density 
functions,  cumulative  distribution  functions,  Tukey  box  plots,  discretized  versions  of  the  density  function, 
pie  charts,  density  functions  at  half  height  with  mirror  images,  error  bars,  fractile  graphs,  triangle  plots,  and 
probability  densities  using  vertical  bars  or  dot  densities  are  among  the  tools  used  to  visually  display 
information.  Both  single  and  multi-dimensional  displays  are  presented.  The  chapter  concludes  that  the 
audience  for  the  information  is  the  most  important  factor  in  choosing  a  communication  medium. 

Chapter  10  provides  an  interesting  but  already  dated  case  study  of  uncertainty  modeling.  Guidance 
on  the  construction  of  complex  models  is  presented  in  Chapter  11.  Regulatory  models  and  global  system 
models  are  among  the  large  models  discussed.  Pros  and  cons,  as  well  as  caveats,  for  large  models  are 
presented. 

The  final  chapter  concludes  with  an  admonition  to  be  aware  of  the  limits  to  our  knowledge  and  a 
summary  discussion  of  the  intriguing  idea  of  the  expected  value  of  including  uncertainty  (EVIU).  EVIU  is 
a  concept  developed  at  length  by  Henrion  in  other  works  in  the  early  1980s.  The  notion  of  EVIU  helps  to 
answer  the  question  how  much  better  will  our  decisions  be  if  we  consider  uncertainty  explicitly?  The  chapter 
addresses  the  issue  of  whether  it's  better  to  use  the  best  estimates  of  uncertain  values  and  arrive  at  a  grand 
best  estimate  or  is  it  preferable  to  use  distributions  of  uncertain  values.  The  author  suggests  that  EVIU  be 
used  to  determine  which  approach  is  better. 

Virtually  all  of  this  text  is  derivative  of  the  work  of  others.  That  in  no  way  diminishes  the  value  of 
this  text.  It  is  a  masterful  work  of  pulling  together  a  coherent  treatment  of  a  vast  new  subject  matter.  It  is 
done  with  eminently  clear  prose,  excellent  examples  and  an  extraordinary  list  of  references. 


74 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography 


G14:  ‘‘A  Method  for  Treating  Dependencies  Between  Variables  in  Simulation  Risk-Analysis 

Models”.  By  P.D.  Newendorp.  Journal  of  Petroleum  Technology:  Official  Monthly  Publication 
of  the  Petroleum  Branch.  American  Institute  of Mining  and  Metallurgical  Engineers.  Vol.  28,  No. 
10,  October  1976.  pp.  1145-1150. 

Applicability 

This  was  one  of  the  few  articles  found  that  addressed  dependencies  between  variables  in  risk 
analysis.  It  is  anticipated  that  dependencies  among  and  between  variables  in  environmental  resource 
planning  will  become  a  significant  issue.  Understanding  the  complex  ways  environmental  variables 
interrelate  has  always  and  will  continue  to  represent  a  significant  issue  in  environmental  resources  planning. 

This  is  a  useful  article.  It  introduces  a  method  for  handling  dependencies  in  a  Monte  Carlo  process. 
This  would  seem  to  be  a  rather  difficult  task  at  first.  Newendorp  offers  a  technique  so  simple  and  so  obvious 
that  he  makes  you  feel  smart  to  see  it  so  clearly.  When  this  article  was  written  in  1976  this  must  have  been 
singular.  Today,  commercial  Monte  Carlo  software  like  @RISK  and  Crystal  Ball  do  these  calculations  for 
you,  although  the  algorithms  used  by  these  companies  are  generally  unknown  to  the  user.  This  paper  is  well 
written  and  fun  to  read,  but  we’re  not  sure  there  is  a  need  for  it  unless  you  are  writing  your  own  program  and 
want  an  easy  way  to  handle  dependencies  among  variables. 

Review 


Newendorp  notes  that  most  of  the  literature  on  Monte  Carlo  simulations  is  written  assuming 
variables  are  independent  of  one  another.  He  notes  that  there  are  many  circumstances  in  risk  and  uncertainty 
analysis  in  which  variables  are  related  in  a  dependent  fashion.  The  paper  discusses  two  points  relating  to 
variable  dependencies.  First,  he  offers  a  technique  for  determining  if  random  variables  are  dependent. 
Second,  he  shows  how  to  modify  the  sampling  procedures  on  each  iteration  of  a  simulation  to  account  for 
observed  partial  dependencies  between  random  variables. 


Acknowledging  that  there  are  many  measures  of  correlation  the  author  presents  a  method  that  is 
“easier”  and  “eliminates  a  lot  of  statistical  theory”.  His  suggestion  is  simply  to  plot  one  variable  against 
another  and  look  at  them.  A  complete  dependency,  such  as  a  straight  line,  can  be  handled  by  estimating  the 
function  that  describes  the  dependency.  No  relationship  means  independent  variables.  Between  these  two 
extremes  are  the  partial  dependencies  Newendorp  addresses.  A  boundary  is  drawn  around  the  cross-plot  of 
two  variables  to  include  all  observed  X-Y  pairs.  The  variability  in  Y  is  expressed  as  a  function  of  X  in  a  way 
that  allows  the  boundary  to  change  as  X  changes.  Newendorp  uses  a  normalized  probability  distribution  that 
represents  all  possible  Y  distributions  in  a  dimensionless  form.  He  uses  a  simple  transformation: 

^  ~  Ymin  X  (^max  X  "  ^min  x)(Ynorm) 

Reading  the  equation  of  the  lower  and  upper  bound  of  Y  as  a  function  of  X  is  the  key  to  this  simple  process. 
Once  the  value  of  X  is  selected  by  usual  methods  it  can  be  used  to  estimate  a  value  for  Y. 


75 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography _ _ _ _ 

Newendorp  cautions  that  lack  of  data  is  not  sufficient  reason  for  ignoring  possible  dependencies 
between  variables.  He  advocates  the  use  of  “what  if’  cases  to  define  the  boundaries  of  the  dependency.  The 
article  provides  two  extended  tables  with  figures  that  alone  are  sufficient  to  explain  his  technique  in  enough 
detail  to  adapt  it  to  any  programming  language  that  has  a  simple  random  number  generator. 
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G15:  The  Risk  Assessment  of  Environmental  and  Human  Health  Hazards;  A  Textbook  of  Case 
Studies.  Edited  by  Dennis  J.  Paustenbach.  New  York:  John  Wiley  &  Sons,  1989.  pp.  xx,  1155. 
Thirty-four  articles  in  eight  sections.  ISBN  0-471-84998-7  hardcover. 

Applicability 

The  title  of  this  and  many  other  books  might  suggest  a  great  deal  of  work  has  already  been  done  in 
the  areas  of  primary  interest  to  Corps  of  Engineers  personnel  involved  in  environmental  resource  planning. 
Unfortunately,  that  may  not  be  so.  Environmental  risk  assessment  or  health  risk  assessment,  as  it  is  also 
known,  refers  to  "the  process  or  procedure  used  to  estimate  the  likelihood  that  humans  or  ecological  systems 
will  be  affected  adversely  by  a  chemical  or  physical  agent  under  a  specific  set  of  conditions"  (p.  27).  Risk 
analysis  as  currently  practiced  by  the  Corps  refers  to  a  more  diverse  number  of  unwanted  events,  few  of 
which  are  chemical  agents.  This  book  is  an  excellent  documentation  of  the  environmental  risk  assessment 
literature. 

The  interested  reader  will  find  its  sections  on  Basic  Principles,  Assessing  Water  Contaminants, 
Assessing  the  Risks  to  Wildlife,  and  Risk  Management  particularly  helpful.  Most  of  the  articles  will  require 
the  reader  to  find  applications  by  reasoning  through  analogy.  This  book  is  worthwhile  for  its  references 
alone. 

Review 


Tome  has  rarely  been  used  more  accurately  to  describe  a  book  than  it  is  used  here  to  describe 
Paustenbach's  two  year  effort  involving  over  50  top  scientists  in  the  field  of  health  risk  assessment.  Risk 
assessment  as  practiced  by  those  concerned  with  health  risks  as  defined  above  is  divided  into  four  major 
steps:  hazard  identification,  dose-response  assessment,  exposure  assessment,  and  risk  characterization. 

"Hazard  identification. ..is  defined  here  as  the  process  of  determining  whether  human 
exposure  to  an  agent  could  cause  an  increase  in  the  incidence  of  a  health  condition  (cancer, 
birth  defect,  etc.)  or  whether  exposure  by  a  nonhuman  receptor,  for  example,  fish,  birds,  or 
other  wildlife,  might  adversely  be  affected.  It  involves  characterizing  the  nature  and 
strength  of  the  evidence  of  causation... 

Dose-response  assessment  is  the  process  of  characterizing  the  relation  between  the  dose  of 
an  agent  administered  or  received  and  the  incidence  of  an  adverse  health  effect  in  exposed 
populations  and  estimating  the  incidence  of  the  effect  as  a  function  of  exposure  to  the 
agent... 

Exposure  assessment  is  the  process  of  measuring  or  estimating  the  intensity,  frequency,  and 
duration  of  human  or  animal  exposure  to  an  agent  currently  present  in  the  environment  or 
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of  estimating  hypothetical  exposures  that  might  arise  from  the  release  of  new  chemicals  into 

the  environment  (p.  30). 

Risk  characterization  is  the  process  of  estimating  the  incidence  of  a  health  effect  under  the 

various  conditions  of  human  or  animal  exposure  described  in  the  exposure  assessment.  It 

is  performed  by  combining  the  exposure  and  dose-response  assessment”  (p.  32). 

This  four  step  process  provides  the  theoretical  underpinning  for  most  of  the  articles  found  in  this  book. 

In  the  Basic  Principles  section  (seven  articles),  the  reader  can  find  a  general  introduction  as  well  as 
some  useful  examples.  The  first  article,  "A  Survey  of  Health  Risk  Assessment"  provides  a  thorough 
overview  of  the  basic  model  in  all  of  its  complexity.  It  is  a  modem,  up-to-date  and  thoroughgoing 
introduction  to  the  basic  model  and  issues  of  environmental  risk  assessment.  The  Cox  and  Ricci  article 
provides  a  good  discussion  of  basic  concepts  in  quantifying  human  health  risks  through  presentation  and 
discussion  of  numerous  examples.  Subsequent  articles  address:  epidemiology,  the  science  of  disease 
distribution;  time-response  relationships;  physiological  pharmacokinetic  models,  the  next  generation  of  dose- 
response  models;  superfund  risk  assessments;  and  a  methodology  for  assessing  risks  to  humans  and  wildlife 
presented  by  contaminated  soils. 

Section  B,  Assessing  Water  Contaminants,  (four  articles)  tends  to  focus  on  specific  chemical  agents 
(e.g.,  1,1,  1-trichloroethane  and  cationic  surfactants).  In  so  doing,  however,  they  provide  an  informative 
discussion  of  contamination  and  migration  of  contaminants  as  well  as  methods  used  to  apply  the  various 
steps  of  the  four  step  model.  There  is  considerable  scientific  detail  on  toxicity,  metabolism,  and  target  organ 
toxicity. 

Assessing  Hazardous  Waste  Sites  is  the  third  section  (five  articles).  The  articles  include  an  actual 
risk  assessment  of  a  pesticide  production  facility.  A  site  specific  risk  assessment,  also  called  an 
endangerment  assessment,  is  provided  for  a  mine  tailings  site.  This  latter  article  provides  a  good 
introduction  to  the  terminology  and  methodology  of  the  superfund  analytical  process. 

The  Maxim  article,  "Problems  Associated  With  the  Use  of  Conservative  Assumptions  in  Exposure 
and  Risk  Analysis"  is  a  particularly  insightful  article  for  any  analyst  or  decision  maker  who,  because  of 
uncertainty,  complexity  or  politics,  has  been  tempted  to  make  a  conservative  assumption  about  inherent  risks. 
Though  the  article  is  oriented  specifically  toward  exposure  analysis  its  seven  suggestions  for  improving  risk 
and  exposure  analyses  are  sufficiently  broad  and  general  to  be  of  use  in  any  situation.  The  other  articles 
round  out  the  treatment  of  the  four  step  model  when  applied  to  hazardous  waste  sites. 

The  fourth  section  is  Assessing  Air  Contaminants  (three  articles).  The  Gammage  and  Travis  article 
is  one  of  many  in  this  book  that  treats  the  low-dose  extrapolation  problem.  One  of  the  principle  analytical 
difficulties  encountered  in  the  health  risk  assessment  field  is  how  to  use  data  from  high  dose  exposures  on 
selected  animal  species  to  infer  knowledge  about  lower  doses  and  effects  on  other  animal  or  human  species. 
Such  issues  may  have  a  parallel  in  the  consideration  of  species  survival  in  environmental  resource  studies. 
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The  Harley  article  provides  a  good  example  of  a  risk  assessment  for  lung  cancer  risk.  It  is  particularly  strong 
in  presenting  risk  projection  models. 

Section  E,  Assessing  Occupational  Hazards,  (five  articles)  presents  articles  particularly  useful  for 
understanding  the  dose-response  and  exposure  assessment  steps  of  the  risk  assessment  process  that  drives 
this  volume.  The  hazard  assessment  discussions  of  these  articles  remain  their  strong  points,  however.  The 
focus  is  on  toxicity  levels  for  humans  of  various  chemical  agents.  The  use  of  animal  data  and  next 
generation  physiological  pharmacokinetic  models  in  different  articles  provide  a  view  of  the  many  approaches 
and  problems  with  assessing  toxicity  of  agents  for  humans. 

Assessing  Potential  Hazards  to  Consumers  (two  articles)  is  the  title  of  he  next  brief  section.  Methyl 
mercury  in  fish  and  the  presence  of  di-2-ethylhexyl  phthalate  (DEHP)  in  baby  pacifiers  and  toys  are  the 
subjects  of  two  risk  assessments. 

Section  G,  Assessing  the  Risks  to  Wildlife  (three  articles)  turns  the  emphasis  from  humans  to  wildlife. 
Ecotoxicology,  assessing  unreasonable  risks  to  the  environment,  is  a  field  of  inquiry  of  considerable  interest 
to  the  Corps  in  its  dredging  program.  It  is  a  field  of  inquiry  that  may  become  of  increasing  interest  as 
involvement  with  environmental  resources  planning  develops.  Ecological  risk  assessment  is  complicated 
by  the  fact  that  we  are  concerned  with  a  myriad  of  species  and  not  just  a  single  chemical  but  a  host  of 
chemical  and  non-chemical  elements  that  interact  in  complex  ways  in  aquatic  and  terrestrial  communities. 

The  Rand  article  addresses  some  general  ecotoxicology  concerns  such  as  chemical  identification, 
metabolism  and  mammalian  toxicology,  and  the  environmental  fate  of  chemical  agents.  This  is  done  in  the 
context  of  a  case  study  of  carbofuran's  effect  on  birds. 

Keenan  et  al.  examine  the  effect  of  sludge  produced  by  a  pulp  and  paper  mill  used  for  mine 
reclamation  on  the  health  of  animals.  The  article  illustrates  the  calculation  of  a  lifetime  average  daily  dose 
(LADD)  for  dermal  contact,  inhalation,  and  soil  uptake;  three  routes  of  exposure  to  2,  3,  7,  8- 
tetrachlorodibenzo-p-dioxin  (2,  3,  7,  8-TCDD)  are  considered.  The  article  provides  example  calculations 
showing  how  the  LADD  can  be  used  to  estimate  TCDD  uptake  through  consumption  of  game  that  have 
ingested  TCDD.  It  is  one  of  the  better  articles  for  illustrating  cancer  risks  for  individual  scenarios  of 
exposure  to  chemical  agents. 

The  Gandy  article  presents  a  hazard  assessment  for  a  bald  eagle  population  near  a  superfund  site. 
The  study  struggles  with  the  lack  of  critical  data  in  a  manner  with  which  many  Corps  analysts  will  identify. 
The  case  study  comprises  a  site  overview,  identification  of  indicator  chemicals,  and  a  hazard  assessment. 
It  is  one  of  the  few  articles  in  the  entire  collection  to  address  potential  hazards  to  wildlife  habitat. 

The  articles  in  the  last  section.  Risk  Management,  as  a  body  provide  a  good  review  of  the  basic 
management  issues  encountered  in  quantitative  risk  assessments  of  health  hazards.  The  findings  are 
sufficiently  general  to  be  of  considerable  value  to  any  risk  manager.  The  articles  generally  provide  good 
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examples  of  the  quantitative  risk  assessment  techniques  criticized  in  the  Fischer  article  (reference  G8) 
reviewed  elsewhere  in  this  literature  review.  The  Cox  and  Ricci  article  provides  a  helpful  introduction  to 
concepts  like  voluntariness  of  risk,  equity  of  risk,  and  procedural  legitimacy.  They  conclude  quite  rightly, 
we  believe,  that  acceptability  of  risk  is  context  sensitive.  What  is  acceptable  risk  today  may  no  longer  be 
tomorrow. 

The  Krewski  et  al.  article  provides  a  framework  for  evaluating  risk  management  strategies  that 
parallels  the  Principles  and  Guidelines  planning  framework  in  spirit,  though  not  in  fact.  The  framework  is 
illustrated  with  an  example  that  is  a  bit  underdeveloped. 

The  article  by  Ames  et  al.  confronts  the  interesting  problem  of  how  to  rank  possible  carcinogenic 
hazards.  They  develop  a  scale,  not  of  risks  to  humans,  but  rather  of  priorities  for  concern,  an  approach  that 
could  be  profitably  mimicked  in  other  contexts  as  well. 

The  final  article  on  nonpessimistic  risk  analysis  returns  to  Maxim's  topic  of  conservative  risk 
estimation  from  the  viewpoint  of  risk  management  rather  than  risk  assessment.  It  likewise  finds  substantial 
problems  with  worst-case  assumptions  standing  in  for  better  analysis. 
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G16:  "Reducing  Risk:  Setting  Priorities  and  Strategies  for  Environmental  Protection".  By  Science 
Advisory  Board  (A-101),  SAB-EC-90-021.  The  Report  of  the  Science  Advisory  Board:  Relative 
Risk  Reduction  Strategies  Committee  to  the  U.S.  Environmental  Protection  Agency. 
Washington,  D.C.:  USEPA,  September  25, 1990.  pp.  v,  26. 

Applicability 

This  document  was  selected  to  assure  an  awareness  of  EPA  policies  regarding  environmental 
protection.  It  provides  a  summary  of  the  EPA's  recommendations  and  findings  for  its  risk  analysis  program 
as  of  1990.  Though  somewhat  dated,  this  is  still  a  valuable  reference  document  for  Corps  personnel 
interested  in  learning  from  the  experience  of  other  federal  agencies.  This  is  a  short  and  easy-to-read 
document  containing  findings  are  specific,  helpful  and  directive.  Though  the  recommendations  are  less 
applicable  to  the  Corps  because  of  differing  agency  missions,  they  may  still  facilitate  awareness  and 
understanding  of  EPA  procedures. 

Review 


Despite  annual  public  and  private  expenditures  to  protect  the  environment  on  the  magnitude  of  $100 
billion,  many  national  environmental  goals  have  not  been  attained.  In  retrospect  our  efforts  have  been 
inconsistent,  uncoordinated,  and  less  effective  than  possible.  The  fragmented  nature  of  our  approach  to 
environmental  problems  is  evident  in  our  laws,  programs  and  tools,  all  of  which  have  been  rather  ad  hoc  and 
somewhat  shortsighted.  It  is  critical  that  U.S.  environmental  policy  must  become  more  integrated  and  more 
focused  on  opportunities  for  environmental  improvement  than  it  has  been  in  the  past. 

"One  tool  that  can  help  foster  the  evolution  of  an  integrated  and  targeted  national 
environmental  policy  is  the  concept  of  environmental  risk.  Each  environmental  problem 
poses  some  possibility  of  harm  to  human  health,  the  ecology,  the  economic  system,  or  the 
quality  of  human  life.  That  is,  each  problem  poses  some  environmental  risk.  Risk 
assessment  is  the  process  by  which  the  form,  dimension,  and  characteristics  of  that  risk  are 
estimated,  and  risk  management  is  the  process  by  which  the  risk  is  reduced.  The  concept 
of  environmental  risk,  together  with  its  related  terminology  and  analytical  methodologies, 
helps  people  discuss  disparate  environmental  problems  with  a  common  language.  It  allows 
many  environmental  problems  to  be  measured  and  compared  in  common  terms,  and  it 
allows  different  risk  reduction  options  to  be  evaluated  from  a  common  basis.  Thus  the 
concept  of  environmental  risk  can  help  the  nation  develop  environmental  policies  in  a 
consistent  and  systematic  way"  p.  2. 

EPA,  in  this  document,  describes  itself  as  a  reactive  agency  that  has  seen  problems  as  separate. 
There  has  been  little  correlation  between  work  effort  and  resources  dedicated  to  environmental  problems 
and  the  risks  posed  by  those  problems.  The  question  of  relative  risk  was  faced  for  the  first  time  in  1986. 
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About  75  managers  and  staff  compared  the  relative  risks  of  3 1  environmental  problems.  They  addressed 
the  residual  risks  after  then-current  controls  were  implemented.  The  results  of  that  effort  were  published  in 
Unfinished  Business:  A  Comparative  Assessment  of  Environmental  Problems.  This  report  was  a  watershed 
event  for  EPA,  but  it  also  showed  that  "business-as-usual"  was  no  longer  an  option  for  dealing  with 
environmental  risks  of  the  1990s  and  beyond. 

The  Science  Advisory  Board  (SAB)  formed  the  Relative  Risk  Reduction  Strategies  Committee 
(RRRSC)  in  response  to  the  Unfinished  Business  report.  The  RRRSC  was  divided  into  three  subcommittees: 
Ecology  and  Welfare;  Human  Health;  and  Strategic  Options.  The  combined  efforts  of  the  RRRSC  were  to: 
1)  critically  review  Unfinished  Business',  2)  merge  the  evaluations  of  cancer  and  non-cancer  risk  and 
ecological  and  welfare  risks;  3)  provide  strategies  for  reducing  environmental  risk;  and,  4)  develop  a  long¬ 
term  strategy  for  improving  risk  assessment/analysis  methodologies.  This  report  highlights  the  most 
important  policy-oriented  findings  and  recommendations  from  the  three  subcommittee  reports. 

Ten  recommendations  are  offered  on  policy  matters  that  include  targeting  future  efforts,  expanding 
the  scope  of  risk-based  priorities  to  strategic  planning  and  budgetary  processes,  among  other  matters.  Eight 
major  findings  are  reported  in  the  document.  These  were:  1)  the  importance  of  Unfinished  Business,  2) 
problems  in  ranking  risks;  3)  the  extraordinary  value  of  natural  ecosystems;  4)  time,  space,  and  risk,  5)  the 
links  between  risk  and  choice;  6)  public  perceptions  of  risk;  7)  relatively  high-risk  environmental  problems; 
and,  8)  strategy  options  for  reducing  environmental  risk. 

The  findings  provide  a  useful  survey  of  the  basic  problems  Corps  analysts  can  expect  to  encounter. 
They  range  from  things  like  lack  of  data  and  methodological  inadequacies  to  the  inadequacy  of  the 
willingness  to  pay  concept  in  risk  analysis.  The  findings  address  issues  of  intrinsic,  moral  values  that  are 
quite  different  from  the  P&G  planning  framework  the  Corps  uses. 

The  document  is  only  a  summary  but  it  is  an  effective  one.  It  provides  a  snapshot  of  the  state  of  EPA 
risk  analysis  policy  as  of  1990.  The  mission  of  EPA  is  different  from  that  of  the  Corps,  but  the  lessons 
learned  by  EPA  can  be  very  helpful  to  Corps  analysts  and  decision  makers.  We  find  this  and  other  EPA 
documents  useful  reading  for  Corps  personnel.  An  awareness  of  the  experience  of  other  agencies  can  be  an 
important  component  of  the  development  of  the  Corps'  risk  analysis  procedures  for  environmental  resources 
planning. 
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APPLIED  RISK  ANALYSIS  LITERATURE 


Al:  “Risk  Analysis  as  a  Project  Management  Tool".  By  B.  Allardyce  and  G.M.  Pottinger.  The 

Journal  of  Canadian  Petroleum.  Vol.  24,  No.  2,  pp.  54-59,  April,  1985. 

Applicability 

This  article  provides  a  good  example  of  a  general  risk  analysis  as  it  was  conducted  in  the  1980s.  The 
authors  used  Monte  Carlo  simulation  and  sensitivity  analysis,  still  two  of  the  more  popular  tools  for  risk 
analysis.  Their  method  is  rather  straightforward  and  simple.  They  did  several  independent  risk  analyses  for 
project  components  and  used  a  fault  tree  to  combine  the  results.  Their  decision  criterion  was  the  rate  of 
return,  a  technique  used  more  by  the  private  sector  than  the  government.  The  paper  is  easy  to  read.  We  had 
hoped,  based  on  the  title,  for  more  specific  guidance  on  how  risk  analysis  is  used  by  managers.  The  paper, 
though  reasonable  for  its  time,  is  now  dated  and  is  not  particularly  recommended. 

Review 


The  Arctic  Pilot  Project  was  to  have  been  a  marine  transportation  system  conceived  to  exploit  the 
large  gas  reserves  of  Canada’s  Northern  Archipelago.  Its  total  cost  was  estimated  to  be  $3  billion  (1983 
Canadian).  The  treatment  of  economic  and  technical  risks  was  a  significant  part  of  the  overall  project 
management  function. 

In  order  for  project  management  to  deal  effectively  with  risk  a  project  specific  risk  analysis 
methodology  was  developed.  The  risk  analysis  was  designed  in  order  to  provide  backers  of  the  project  with 
information  about  the  likely  economic  viability  of  the  project.  Several  independent  risk  analyses  were 
undertaken  for  the  logistics,  facilities,  shipping  and  sea-lift  components  of  the  project.  It  was  then  necessary 
to  integrate  these  studies  in  a  fashion  that  summarizes  the  risk  potential  of  the  entire  project. 

The  rate  of  return  was  identified  as  the  index  variable  to  be  used  to  represent  risks  to  the  overall 
project.  This  indicator  was  judged  to  be  the  best  indicator  of  the  project's  overall  success.  The  analysis  did 
not  include  consideration  of  risk  to  environmental  resources.  It's  emphasis  appears  to  be  on  the  project's 
economic  performance,  and,  as  a  result  of  which,  the  project  was  abandoned. 

A  fault  tree  was  used  to  delineate  the  sources  of  risk  which  were  combined  according  to  the  fault  tree 
structure.  Risks  were  quantified  in  the  form  of  cumulative  probability  distributions  (cdf)  and  the  analysis 
relied  on  simulations  using  Monte  Carlo  procedures  and  sensitivity  analysis.  Results  are  presented  in  terms 
of  the  cdf  for  rate  of  return. 
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The  analysis  relied  most  heavily  on  fault  tree  analysis  and  "risk  centre  identification  .  Risk  centres 
identified  were  capital  costs,  operating  costs,  estimates  of  escalation  rates,  market  price  estimates,  and 
overall  project  production.  Key  variables  were  identified  for  the  risk  centres  and  were  the  focus  of 
subsequent  sensitivity  analyses. 

The  authors  suggest  their  method  is  a  general  one.  The  steps  include;  1)  establishing  a  project 
breakdown  structure;  2)  identifying  risk  centres;  3)  constructing  a  fault  tree;  and  4)  calculating  probability 
distributions  for  all  pertinent  available  input  data.  The  simple  act  of  creating  the  risk  model  is  valuable  for 
its  identification  of  the  project's  most  sensitive  components. 
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A2:  “Problems  of  Risk  Assessment  in  Water  Resources  Management.”  By  Richard  C.  Allison  and 

Roger  Durand.  The  Environmental  Professional.  Vol.  13. 1991.  pp.  326-330. 

Applicahility 

In  this  paper  the  authors  argue  the  importance  of  risk  assessment  and  risk  management  for  water 
resources.  Their  view  is  a  big  picture  one.  Though  there  is  little  that  can  be  called  new  or  revealing  in  this 
article,  it  does  affirm  the  importance  of  the  risk  analysis  work  the  Corps  is  involved  with  in  the  management 
of  the  Nation’s  water  and  environmental  resources. 

Review 


The  need  to  assess  risks  to  water  resources  and  to  manage  those  risks  properly  has  never  been  more 
evident.  The  authors  offer  examples  of  drought,  hospital  wastes  on  the  shoreline,  and  other  recent  headline 
events  as  proof  of  their  assertion.  Despite  the  mention  of  flood  and  drought,  their  focus  seems  clearly  fixed 
on  human  health  issues.  With  such  an  urgent  need,  the  time  for  proper  risk  assessment  is  now.  Despite 
important  developments  in  risk  assessment,  including  scholarly  and  popular  attention,  better  analytical 
methods,  and  more  effective  communication,  they  conclude  water  resource  analysts  ability  to  assess  risks 
remains  remarkably  limited. 

Allison  and  Durand  offer  eight  major  interrelated  problems  that  constrain  our  ability  to  do  proper 
risk  analysis.  These  problems  inhibit  the  resolution  of  debate  with  respect  to  water  health  and  safety.  The 
problems  are:  1)  failure  to  distinguish  risks  to  the  environment  from  risks  to  human  health;  2)  lack  of 
general  agreement  on  what  constitutes  an  adverse  effect  on  human  health;  3)  the  inability  to  specify  cause- 
effect  relationships  for  certain  health  and  safety  outcomes;  4)  lack  of  a  dose-response  threshold  for  many 
hazardous  materials;  5)  great  variation  in  the  susceptibility  of  human  beings  to  the  effects  of  exposure  even 
when  there  is  a  dose-response  threshold;  6)  limited  understanding  of  the  toxicological  consequences  of  many 
substances  in  wide  use;  7)  inability  to  obtain  valid,  reliable,  and  sufficiently  precise  measurements  of  crucial 
risk  variables;  and  8)  difficulty  defining  acceptable  risk. 

The  authors  offer  examples  of  how  these  problems  limit  the  debate  on  water  resources  for 
recreational  water,  drinking  water  and  irrigation  water.  For  example,  “At  present,  there  is  virtually  no 
reliable  and  valid  epidemiological  evidence  to  support  a  quantitative,  microbiological  standard  based  on  fecal 
conforms  forjudging  recreational  water  safety.”  A  risk  assessment  by  the  Wastewater  Division  of  the  City 
of  Colorado  Springs  is  put  forth  as  a  model  study. 

What  can  managers  do  in  the  face  of  these  seemingly  intractable  problems?  First,  abandon  any  idea 
that  there  exists  a  general-purpose,  uniform  “off-the-rack”  approach  to  risk  assessment  that  will  be  good  for 
all  occasions.  Second,  conduct  environmental,  epidemiologic  assessments  that  are  tailored  to  your  own  risk 
situations  and  to  the  variables  of  your  own  unique  setting.  Third,  concentrate  on  developing  what  has  been 
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termed  “policy  for  error  variance”,  an  option  for  coping  with  or  adjusting  to  the  uncertain  or  unpredictable. 
This  is  an  idea  similar  in  spirit  to  the  adaptive  management  strategy  the  Corps  has  proposed  for  ecosystem 
restoration.  Fourth,  go  to  work  immediately  on  fostering  improved  communications  between  the  “makers” 
and  the  “takers”  of  risk. 


86 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography 


A3:  "On  Quality,  Peer  Review,  and  the  Achievement  of  Consensus  in  Probabilistic  Risk  Analysis." 

By  G.  Apostolakis,  B.J.  Garrick,  and  D.  Okrent.  Nuclear  Safety.  Vol.  24,  No.  6,  November- 
December,  1983.  pp.  792-800. 

Applicability 


This  article  was  selected  for  review  for  several  reasons.  First,  it  presents  some  of  the  thoughts  of 
the  Nuclear  Regulatory  Commission  (NRC),  and  it  is  important  to  proceed  with  some  awareness  of  what 
other  agencies  are  doing  in  the  area  of  risk  analysis.  There  are  several  publications  referenced  in  this  article 
not  reviewed  elsewhere  in  this  report  that  would  appear  to  be  well  worth  the  time  required  to  become 
familiar  with  them.  Second,  we  find  the  title  suggests  many  issues  that  will  be  important  to  Corps  analysts 
involved  in  environmental  resource  risk  analysis.  The  quality  of  the  work  is  always  going  to  be  a  concern. 
We  believe  that  because  of  the  potential  controversy  attending  environmental  issues  both  peer  review  and 
achieving  consensus  are  going  to  be  very  important  issues. 

The  paper  was  rather  general  in  tone  and  as  such  it  serves  as  a  very  good  introduction  to  some  of  the 
work  of  the  NRC  and  to  the  issues  raised  in  the  title.  In  recounting  lessons  learned  it  offers  Corps  planners 
insight  into  issues  that  may  have  taken  months  or  years  to  learn  from  experience.  For  example,  does  the 
Corps  want  a  risk  analysis  paradigm  that  all  would  follow  like  the  NRC  favors  and  EPA  has?  It  does  provide 
some  quality  assurance  but  it  also  could  result  in  finding  square  pegs  in  round  holes.  This  issue  and  others 
like  it  deserve  careful  consideration  by  analysts  seeking  to  establish  reasonable  standards  for  risk  analysis 
in  a  new  area  of  endeavor.  The  article  is  highly  recommended  to  those  interested  in  risk  analysis  paradigms, 
guidance  and  the  like. 

Review 


Several  problems  face  users  of  Probabilistic  Risk  Analysis  (PRA).  First,  how  do  you  put  the 
numerous  and  diverse  elements  of  a  PRA  together  in  a  way  that  does  not  violate  any  rules  of  the  many 
disciplines  involved  in  the  PRA?  Second,  how  do  we  assure  that  independent  groups  can  reproduce 
significant  results?  Third,  how  do  we  reach  consensus  with  problems  that  rely  so  heavily  on  decision  tools 
developed  for  use  by  a  single  person  (for  example,  subjective  probability  theoiy)  when  groups  of  people  try 
to  use  these  theories  and  tools?  In  this  article  the  authors  address  several  issues  that  arise  in  connection  with 
these  problems. 

With  regard  to  quality  assurance,  i.e.,  the  quality  of  the  PRA  itself,  the  authors  note  that  PRA  is  a 
tool  that  can  be  used  or  abused  by  project  opponents.  Furthermore,  when  data  are  lacking  the  PRA  can  be 
legitimately  questioned  by  anyone.  NRC  safety  policy  has  advanced  the  position  that: 
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"One  way  to  improve  the  consistency  of  PRA  results  is  to  provide  some  assurance  that 
analysts  follow  equivalent  procedures,  make  similar  assumptions,  treat  phenomena 
consistently,  and  utilize  a  common  data  base"  (p.  794). 

The  NRC  staff  favors  a  highly  prescribed  PRA  format.  The  National  Reliability  Evaluation  Program 
(NREP)  Procedures  Guide  is  a  step  in  this  direction.  The  benchmarks  established  make  quality  comparisons 
easier.  The  first  step  toward  standardization  suggested  is  to  limit  the  scope  of  the  PRA,  considering  only  the 
most  important  things  first,  gradually  expanding  the  scope  of  the  analysis.  This  is  an  approach  that  could 
aid  the  development  of  a  risk  analysis  strategy  by  the  Corps. 

Proceeding  cautiously  and  with  open  debate  and  discussion  seems  to  be  the  unspoken  principle  for 
developing  PRA  guidelines  for  the  NRC.  Peer  review  is  considered  to  be  an  extremely  important  part  of  a 
PRA.  In  fact,  four  levels  of  review  are  proposed  in  the  PRA  Procedures  Guide.  Study  team  review  of  all 
work  by  an  internal  peer  review  group  is  first.  Methodological  mistakes  are  identified  with  the  most 
confidence  at  this  level.  Review  by  plant  operating  personnel  would  allow  people  most  familiar  with  the 
operation  of  systems  (for  the  Corps  these  might  be  resource  agency  personnel)  to  review  the  work  for 
technical  errors  in  modeling  the  relevant  plant  and  site  characteristics  (perhaps  ecosystems  in  the  Corps 
studies). 

The  third  level  of  review  is  peer  review.  This  is  to  be  done  by  true  peers,  people  not  involved  with 
the  study  but  with  capabilities  equivalent  to  those  who  did  the  study.  This  review  would  concentrate  on  the 
methods,  information  sources,  judgments  and  assumptions.  Management  review  would  concentrate  on  the 
perspective,  scope,  and  suitability  of  the  analysis. 

The  consensus  problem  is  to  be  expected  in  a  PRA,  especially  when  the  subject  matter  is  one  of 
controversy.  Experts  do  not  have  to  agree  on  everything.  Many  aspects  of  probability  theory  and  decision 
theory  have  been  developed  for  analysts  acting  alone.  When  groups  have  to  reach  consensus  on  analyses 
that  rely  on  highly  individualized  tools  (for  example,  subjective  probability  assessments)  disagreement  is 
inevitable.  Where  data  exist  the  interpretation  of  evidence  can  vary  from  expert-to-expert.  The  authors 
argue  that  quality  assurance  is  an  important  part  of  the  process  that  could  lead  to  consensus.  They  see 
standardization  of  methods  as  a  way  to  provide  that  quality  assurance. 
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A4:  “Risk  Assessment  Principles  in  Environmental  Impact  Studies.”  By  M.  Carolota  Arquiaga, 

Larry  W.  Canter,  Deborah  Imel  Nelson.  The  Environmental  Professional.  Vol.  14.  No.  3. 1992. 
pp.  204-219. 

Applicability 

This  paper  is  the  conceptual  godfather  of  the  articles  by  Canter  and  Cairns  (1993),  found  elsewhere 
in  this  review  (reference  AlO).  It’s  primary  purpose  is  to  establish  that  EISs  were  not  doing  what  they  were 
supposed  to  do  with  regard  to  the  analysis  of  human  health  impacts.  This  it  does  well.  In  its  second  part, 
it  establishes  a  need  for  an  environmental  health  assessment  methodology  and  suggests  such  a  generic 
methodology. 

The  article  is  of  some  historical  interest.  It  offers  very  little  of  immediate  value  to  Corps  analysts 
developing  risk  analysis  methodologies  for  environmental  resources  planning.  Like  many  other  articles 
reviewed  in  this  report,  it  was  chosen  solely  on  the  strength  of  its  title,  which  simply  could  not  be 
overlooked. 

Review 


This  paper  comprises  two  major  parts.  In  the  first,  it  establishes  that  EISs  have  not  done  the  human 
health  impact  assessments  that  policy  and  law  require.  In  the  second  part,  the  authors  offer  a  generic 
assessment  methodology. 

The  introduction  begins  by  pointing  out  the  significance  of  the  National  Environmental  Policy  Act 
(NEPA)  of  1970.  The  environmental  impact  assessment  (EIA)  procedure  required  by  NEPA  has  now  been 
adopted  by  more  than  75  countries  around  the  world.  The  authors  argue  that  two  topics  have  received 
inadequate  attention  in  the  EIA  process.  They  are  “..the  minimal  focus  that  human  health  impacts  typically 
are  given  in  EISs  and  the  lack  of  quantitative  assessment  methods  for  the  prediction  and  assessment  of 
human  health  impacts.”  Interestingly,  many  authors  encountered  in  this  review  have  argued  that  human 
health  effects  have  been  emphasized  almost  to  the  exclusion  of  natural  ecosystems.  These  authors  argue  the 
exact  opposite. 

The  authors,  all  from  the  University  of  Oklahoma,  reviewed  39  EISs  prepared  in  the  U.S.  between 
1980  and  1990.  This  was  not  a  random  sample  of  EISs.  Using  a  set  of  review  questions,  the  authors  set 
about  a  systematic  review  of  these  EISs.  In  their  discussion,  the  authors  describe  many  of  the  problems  they 
encountered,  including  such  fundamental  issues  as  deciding  what  is  meant  by  health.  The  review  included 
four  flood/hurricane  surge  protection  projects,  one  reservoir,  one  marina,  and  one  river  dredging  project. 
The  remaining  32  EISs  covered  a  wide  variety  of  construction  and  disposal  projects. 


89 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography _ _ 

Health  impacts  were  divided  into  physical,  chemical,  radiological,  and  biological.  The  number  of 
EISs  including  each  of  these  types  of  impacts  was  identified.  An  EIS  could  include  more  than  one  type.  Of 
38  EISs  with  physical  health  impacts,  19  addressed  these  impacts,  three  of  them  in  a  quantitative  manner. 
Thirty-three  chemical  impact  EISs  saw  20  addressed,  three  of  them  quantitatively.  Nine  of  ten  radiological 
health  impacts  were  addressed,  none  of  them  quantitatively.  Of  13  biological  health  impacts,  three  were 
addressed,  none  quantitatively.  From  these  results  the  authors  concluded  human  health  impacts  have  been 
given  inadequate  attention  in  most  EISs  since  1980.  Biological  impacts  were  concluded  to  have  received 
the  least  attention. 

Arquiaga  et  al.  offer  ten  reasons  illustrating  the  need  for  a  health  impact  assessment  methodology. 
They  next  offer  a  generic  methodology  that  is  based  on  the  ten  fundamental  activities  conducted  in  a  typical 
EIS.  This  methodology,  summarized  in  the  authors  own  words  follows: 

1)  Preparation  of  description  of  project  and  need  for  project  (note  health-related 
components); 

2)  Review  and  analysis  of  pertinent  institutional  (health-related)  information; 

3)  Identification  of  possible  impacts  (on  human  health)  from  construction  and 
operation  of  project  type  and  alternatives; 

4)  Preparation  of  a  description  of  the  affected  environment  (focus  on  health 
characteristics  and  delivery  systems); 

5)  Prediction  of  impacts  (health  related); 

6)  Assessment  (interpretation)  of  the  predicted  impacts  (based  on  standards/risk 
quantification); 

7)  Identification  and  evaluation  of  mitigation  measures  (to  minimize  undesirable 
health  impacts); 

8)  Selection  of  proposed  action  from  the  alternatives  being  evaluated  (include  health 
impacts  as  a  decision  factor); 

9)  Preparation  of  written  documentation  of  the  EIA  process  (incorporate  health  focus); 

10)  Monitoring  of  environmental  impacts  during  project  construction  and  operation 
(include  health  indicators  and  measures  of  health  hazard). 
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A5:  Ecological  Risk  Estimation.  By  Steven  M.  Bartell,  Robert  H.  Gardner,  and  Robert  V.  O’Neill. 

Boca  Raton:  Lewis  Publishers,  1992.  pp.  252.  Eight  chapters.  ISBN  0-87371-163-7. 

Applicability 

This  is  a  book  for  biologists  skilled  in  the  techniques  of  ecological  risk  analysis  or  for  skilled  risk 
analysts  seeking  a  better  understanding  of  ecological  risk  in  aquatic  systems.  It  is  not  very  mathematical  but 
the  language  is  the  technical  language  of  ecological  risk,  ecology  and  biology.  If  you  are  a  reader  who 
knows  that  an  increased  bioturbation  of  sediments  and  increased  growth  of  certain  oligochaetes  exposed  to 
low  concentrations  of  endrin  under  laboratory  conditions  is  an  example  of  a  hormetic  response,  this  book 
is  for  you.  Although  it  is  not  an  easy  book  for  the  non-professional  to  understand,  there  is  a  wealth  of  useful 
information.  For  the  right  reader  it  is  recommended. 

Review 


Because  our  ability  to  forecast  the  probable  effects  of  novel  chemicals  on  ecological  systems  has  not 
kept  pace  with  either  society's  increasing  chemical  arsenal  or  its  thirst  for  information  about  these  effects, 
there  is  a  need  for  ecological  risk  analysis.  In  general  terms,  ecological  risk  is  an  attempt  to  quantify  the 
potential  danger  posed  by  chemicals  from  a  variety  of  toxicity  tests  that  have  been  devised.  Ecological  risk 
analysis  remains  in  its  formative  stages  primarily  because  of  our  lack  of  data  and  lack  of  understanding  of 
the  function  and  complexity  of  unique  ecological  systems. 

This  book  presents  a  methodology  that  uses  the  results  of  laboratory  toxicity  tests  to  forecast  the 
likelihood  of  measuring  toxic  chemical  effects  on  populations  in  natural  aquatic  systems.  The  objective  of 
ecological  risk  assessment  is  ”..to  use  available  toxicological  and  ecological  information  to  estimate  the 
probability  that  some  undesired  ecological  event  will  occur."  These  ecological  endpoints  can  include  such 
events  as:  local  extinction  of  some  species;  unacceptable  decline  in  the  abundance  of  some  economically 
important  taxa;  unacceptable  increase  in  the  abundance  of  some  undesired  population  or  species; 
unacceptable  change  in  a  fundamental  ecological  process;  and  loss  of  species  habitat. 

The  authors  do  a  credible  job  of  laying  out  the  attributes  of  a  good  risk  methodology  in  a  pragmatic 
fashion.  The  emphasis  is  clearly  on  being  able  to  use  data  currently  being  produced,  though  the  authors’ 
speculation  on  future  directions  for  ecological  risk  analysis  in  the  concluding  chapter  is  not  to  be  missed. 
The  nature  of  the  toxicological  and  ecological  data  used  to  estimate  ecological  risk  in  aquatic  systems  is 
described  at  some  length.  The  discussion  is  comprehensive  but  relatively  advanced.  For  example,  median 
lethal  concentrations  (LC50),  effective  concentrations  (EC 50),  median  effective  dose  (ED5(),  and  median  lethal 
dose  (LD50)  are  all  discussed  as  benchmarks  for  estimating  ecological  effects  of  toxic  chemicals;  but  none 
of  them  are  defined  adequately  for  readers  not  already  familiar  with  the  concepts. 


91 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography _ _ _ _ 

Interspersed  throughout  the  book  are  some  valuable  nuggets  of  information.  The  discussions  on 
legislation  and  sources  of  toxicological  data  are  two  such  examples.  Following  a  discussion  of  the  nature 
of  the  data  and  its  inherent  uncertainties  the  authors  offer  a  standard  water  column  model  as  the  aquatic 
ecosystem  model  of  choice.  This  model  proves  interesting  because  it  simulates  complex  competitive  and 
predator-prey  dynamics  in  the  context  of  a  seasonal,  multidimensional  environment.  The  hypothesis 
underlying  the  development  of  ecological  risk  methods  is  that  an  ecosystem  model,  in  this  case  the  standard 
water  column,  can  be  used  to  extrapolate  the  results  of  laboratory  toxicity  data  into  meaningful  predictions 
of  ecological  effects  in  natural  aquatic  systems.  The  water  column  model  is  useful  because  sublethal  toxic 
effects  can  be  expressed  as  changes  in  the  rates  of  the  physiological  process  equations  for  population 
growth.  Whatever  else  its  appeal,  the  model  is  pragmatic. 

The  water  column  model  also  permits  exploration  of  sublethal  toxic  effects  on  predator-prey  and 
competitive  interactions  among  populations  in  addition  to  the  direct  effects  on  specific  populations.  Yet 
another  advantage  of  the  model  is  that  it  explicitly  represents  a  dynamic  physical-chemical  environment  in 
which  the  simulated  ecological  and  toxological  interactions  occur. 

The  authors  present  a  model  designed  to  use  laboratory  toxicity  data  to  modify  the  aquatic  system 
model  to  forecast  the  ecological  effects  of  toxic  chemicals  in  Chapter  4.  Estimating  exposure  is  identified 
as  the  critical  step  in  any  risk  analysis.  Well-defined  concentration-response  functions  (CRFs)  for  each  of 
the  model  populations  and  each  toxic  chemical  would  be  the  ideal.  Such  data  are  not  to  be  expected.  Linear 
approximations  to  the  CRFs  are  the  rule  and  the  interpolation  issues  are  substantial. 

The  key  to  the  bioassay  simulation  model  is  the  effects  matrix.  Population  mortality  is  interpreted 
as  a  decrease  in  biomass.  These  decreases,  related  to  exposure  levels,  can  be  mathematically  described  by 
disaggregating  the  rate  of  decrease  into  its  physiological  constituent  processes.  The  growth  parameters  are 
modified  by  the  so-called  “E  matrix”.  Each  row  element  defines  the  population-specific  fractional  change 
in  a  growth  parameter  required  to  simulate  the  population's  expected  response  to  the  exposure  concentration 
for  that  matrix.  The  effects  factors  are  assigned  to  a  distribution  with  a  mean  equal  to  itself  and  a  coefficient 
of  variation  of  100%.  One  matrix  is  generated  for  each  exposure  concentration  used  in  the  risk  analysis. 
This  is  the  key  component  of  the  overall  risk  model. 

After  presenting  the  component  pieces  of  an  ecological  risk  model  in  the  early  chapters,  a  risk 
algorithm  bringing  all  the  components  together  is  presented  in  Chapter  5.  The  risk  algorithm  was  applied 
to  a  variety  of  organic  and  inorganic  contaminants  for  several  different  aquatic  systems.  One  example 
showing  effects  of  various  chemical  exposures  on  phytoplankton,  planktivorous  fish,  and  other  groups  is 
presented.  Sensitivity  analyses  are  conducted  using  five  different  scenarios.  The  results  are  expressed  as 
probabilities  of  different  endpoints  being  achieved  under  different  exposure  concentrations  of 
chloroparaffins. 

In  an  evaluation  of  the  risk  forecasting  methodology  the  authors  wrestle  with  numerous 
philosophical  and  pragmatic  issues.  Perhaps  one  of  the  most  generally  applicable  points  they  make  is: 
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"Unproductive  debate  can  be  avoided  by  simply  recognizing  that  the  model  is  invalid  by 
definition.  The  value  of  the  model  is  determined  by  its  facility  for  increasing  the  basic 
understanding  of  the  science  involved,  by  the  model's  ability  to  extend  current  predictive 
powers,  or,  if  lucky,  both"  (p.  164). 

The  results  show  that  the  duration  of  the  simulations  used  to  estimate  risk  should  include  several 
cycles  to  ensure  the  effects  are  not  confined  to  transient  system  behavior.  Attempts  to  evaluate  the  accuracy 
of  the  toxic  effects  predictions  were,  generally,  not  successful,  because  of  a  lack  of  data  and  limitations  of 
current  methodology. 
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A6:  "Endangered  Species  and  Uncertainty:  The  Economics  of  a  Safe  Minimum  Standard".  By 

Richard  C.  Bishop.  American  .Tournal  of  Agricultural  Economics,  pp.  10-18,  February,  1978. 

Applicability 

In  this  paper  Bishop  presents  a  clear,  concise  and  articulate  argument  for  giving  the  benefit  of  the 
doubt  to  preservation  alternatives  over  development  alternatives.  This  article  provides  a  decision  criterion 
that  many  environmental  interests  will  welcome.  Though  the  article  is  somewhat  dated  it  still  offers  a  timely 
contribution  to  the  public  policy  debate.  This  article  can  be  easily  read  by  economists  and  others  and  is 
highly  recommended. 

Review 


The  author’s  objective  may  be  summarized  by  the  following  quote  from  the  opening  of  the  article: 

"Suppose  that  a  hydroelectric  dam  has  been  proposed  which  would  flood  the  last  remaining 
habitat  of  an  endangered  species.  Suppose  further  that  there  is  no  alternative  method  of 
preventing  extinction  of  this  species  except  to  maintain  its  habitat  in  an  unflooded  condition. 

The  goal  of  this  paper  is  to  explore  the  conceptual  issues  which  are  central  to  the 
development  of  an  economic  framework  for  confronting  public  decisions  of  this  type"  (p. 

10). 

This  well-written  paper  performs  two  major  tasks.  First,  it  advances  an  argument  for  how  extinction 
irreversibly  narrows  the  reservoir  of  potential  future  resources,  creating  the  possibility  of  large,  though 
uncertain,  future  social  losses.  Second,  it  offers  the  "safe  minimum  standard"  (SMS)  policy  option  that 
would  maintain  sufficient  population  and  habitat  to  assure  the  survival  of  the  species. 

The  problem  of  irreversibility  and  uncertainty  is  that  "characteristics"  of  the  natural  environment 
become  natural  resources  over  time,  through  changes  in  human  tastes  and  preferences,  income,  population, 
technology,  social  institutions,  and  public  policies.  The  long  run  implications  of  allowing  the  reservoir  of 
species  to  be  reduced  are  uncertain.  We  don't  know  if  or  when  unanticipated  events  will  cause  natural 
phenomena  to  become  valuable  natural  resources.  Oil  was  not  a  resource  until  technology  had  advanced  to 
the  point  where  the  energy  stored  in  it  could  be  harnessed  in  productive  ways.  Potential  extinction  is  an 
important  public  policy  issue,  more  now  than  when  this  article  was  first  written.  We  have  little  basis  for 
judging  which  life-forms  we  can  lose  without  future  social  and  economic  consequences. 

One  solution  to  this  problem  of  irreversibility  and  the  uncertainty  of  future  losses  would  be  to  adopt 
the  safe  minimum  standard.  The  SMS  arises  from  the  minimax  principle  of  game  theory,  i.e.,  society  should 
choose  the  strategy  that  minimizes  maximum  possible  losses.  Suppose  society  has  two  choices:  1)  E,  for 
extinction,  which  means  building  the  dam;  and,  2)  SMS,  not  building  the  dam  and  leaving  the  valley 
unflooded,  protecting  the  habitat.  Further  suppose  there  are  two  states  that  can  occur.  In  state  1  there  are 
no  large  future  losses  if  the  species  become  extinct.  In  state  2  the  species  would  be  worth  a  great  deal  to 
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society,  perhaps  as  a  cure  for  cancer  or  a  new  energy  source.  The  table  below  presents  the  matrix  of  losses 
associated  with  this  situation: 


Strategy 

State  1 

State  2 

Maximum  Loss 

E 

0 

y 

y 

SMS 

X 

x-y 

X 

The  value  of  the  species  in  state  2  is  y;  in  state  1  it  has  no  value.  The  present  value  of  net  benefits  from  the 
dam  is  designated  x.  If  we  do  not  build  the  dam  and  state  1  prevails  we  lose  benefits  of  x.  If  state  2  prevails 
we  lose  the  benefits  x  less  the  gains  y  from  having  protected  the  species. 

In  this  case,  E  should  be  chosen  if  x>y,  SMS  should  be  chosen  if  y>x;  and,  if  x=y  we  are  at  a  point 
of  indifference.  The  authors  point  out  that:  1)  this  criterion  is  extremely  conservative;  2)  the  payoff  matrix 
is  not  known  with  certainty;  and,  3)  distribution  effects  of  the  gains  and  losses  are  ignored.  They  modify 
the  minimax  criterion  by  suggesting  that  we  should  adopt  the  SMS  strategy  unless  the  social  costs  of  doing 
so  are  unacceptably  large.  How  high  must  costs  be  before  they  are  unacceptably  large  is  not  pragmatically 
addressed.  The  authors  suggest  it  is  up  to  society  to  decide. 

The  authors  then  go  on  to  discuss  then  recent  work  by  Resources  for  the  Future  (RFF).  This  includes 
one  of  the  earliest  and  best  discussions  of  problems  one  might  encounter  in  estimating  net  development 
benefits,  B^,  and  net  preservation  benefits,  Bp.  The  basic  problem  is  that  there  are  divergences  between  what 
the  authors  see  as  net  measurable  preservation  benefits,  B’p,  and  Bp.  The  reasons  for  the  potential  divergence 
include  the  existence  of:  1)  option  value;  2)  existence  value;  3)  ambiguity  in  property  rights;  and,  4) 
technological  change. 

Because  of  the  existence  of  these  uncertainties  RFF  concluded  that  public  choices  between 
development  and  preservation  must  proceed  in  three  steps.  First,  B^  must  be  carefully  examined.  If  they  are 
negative,  development  is  not  justified  regardless  of  environmental  impacts.  Second,  we  must  evaluate  B’p 
and  compare  them  to  B^.  If  B'p  >  B^,  development  is  not  justified.  Third,  even  when  B’p  <  B^,  the  existence 
of  option  values,  existence  values,  ambiguous  property  rights  ,  and  technological  change  means  that  Bp  > 
Bd  is  possible.  Give  the  benefit  of  the  doubt  to  preservation,  say  the  authors. 
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A7:  Risk  Assessment  in  Conservation  Biology.  By  M.A.  Burgman,  S.  Person,  and  H.R.  Ackakaya. 

London:  Chapman  &  Hall,  1993.  pp.  ix,  314.  Seven  chapters,  four  appendices.  ISBN  0  412 
35030  0. 

Applicability 

This  book  appears  to  be  particularly  applicable  to  biologists  interested  in  applying  risk  analysis  to 
conservation  biology.  The  writing  style  is  eminently  clear  and  the  organization  is  exceptional.  Critical 
concepts  and  terminology  are  clearly  defined.  Population  models  are  developed  in  a  logical  and  easy-to- 
follow  manner.  Though  the  mathematics  will  be  difficult  for  practitioners  and  analysts  who  have  not  used 
these  skills  regularly,  they  should  present  no  problem  to  anyone  who  has  worked  with  mathematical  models 
at  a  graduate  level.  Finding  the  data  for  the  models,  however,  will  be  a  major  constraint  on  the  use  of  these 
models. 

This  book  could  prove  a  treasure  trove  of  concepts,  ideas,  and  models  for  Corps  analysts  and 
managers  seeking  ways  to  formulate  the  objectives  of  environmental  resource  studies.  Furthermore,  it  will 
be  an  invaluable  aid  to  those  who  will  assess  the  environmental  management  alternatives.  The  authors 
provide  algorithms  for  many  of  the  risk  models.  The  references  are  excellent.  Selective  reading  of  this  book 
is  recommended  for  anyone  who  wants  to  learn  some  of  the  language  of  conservation  biology  and  extinction. 
The  book  is  highly  recommended  for  demographers,  biologists,  and  other  biological  scientists. 

Review 


"Apart  from  a  relative  handful  of  wildlife  managers  based  in  government  institutions  around 
the  world,  most  ecologists  have  had  little  to  do  with  the  management  of  the  populations  they 
study  and  less  to  do  with  decision-making  and  policy  formulation  within  the  bureaucracies 
that  administer  wildlife  management.  For  this  reason,  much  ecological  research  has  little 
direct  relevance  to  many  of  the  immediate  and  pressing  problems  facing  wildlife  managers" 

(p.  272). 

With  these  words  the  authors  capture  the  essence  of  what  may  prove  to  be  an  obstacle  to  the  Corps’ 
environmental  resource  planning  efforts,  at  least  as  related  to  conservation  biology.  According  to  the 
authors,  there  is  a  paucity  of  readily  usable  models  for  assessing  and  managing  the  risks  to  selected 
populations  and  species.  Nonetheless,  this  book  does  a  superb  job  of  describing  the  state-of-the-art  in  risk 
assessment  for  population  survival. 

This  is  a  book  about  how  to  build  stochastic  models  for  risk  assessments  of  population  decline.  The 
models  presented  in  the  book  address  several  common  questions  in  conservation  biology,  for  example,  “How 
will  the  population  change  in  the  future?”;  “What  is  the  risk  the  population  will  become  extinct?”;  “How 
should  a  stocking  program  be  managed?”;  and  “Where  should  re-introductions  be  planned  to  optimize 
results?”. 
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The  book  begins  by  presenting  the  change  in  population  size  as  a  random  walk.  If  you  have  never 
understood  what  a  random  walk  is,  have  no  fear.  A  strength  of  this  book  is  its  clarity. 

"Consider  yourself  walking  parallel  to  a  boundary  fence.  At  every  third  step  you  take  a  pace 
sideways,  either  toward  or  away  from  the  fence.  You  decide  which  direction  to  take  after 
flipping  a  coin.  If  it  is  heads  you  move  towards  the  fence  and  if  it  is  tails  you  move 
away"(p.  7). 

Using  this  analogy  to  the  random  walk  the  authors  effectively  introduce  the  basic  terminology  of  the  book. 
Quasi  extinction  risk  is  the  is  the  chance  that  a  population  will  fall  below  a  small  population  threshold  that 
may  be  unacceptable  for  conservation,  management,  economic,  or  aesthetic  purposes.  The  authors  suggest 
the  best  approach  to  risk  assessment  is  to  quantify  the  probability  of  extinction  within  a  given  time  period. 
The  sources  of  uncertainty  that  contribute  to  change  in  population  abundance  can  be  classified  as  phenotypic 
(variation  between  individuals  within  the  same  population),  demographic  (variation  in  the  average  chances 
of  survivorship  and  reproduction),  environmental  (unpredictable  change  in  the  environment  over  time),  and 
spatial  (variation  in  environmental  conditions  between  spatially  separated  patches  of  habitat). 

Unstructured  or  single-dimensional  models  are  models  that  treat  the  entire  population  as  monolithic. 
They  ignore  age  differences,  life  history  stages,  and  different  spatial  patterns.  The  first  models  discussed 
are  linear  models  of  this  type.  A  birth-and-death  model  for  white  rhinos  is  developed  and  made  successively 
more  complex  from  a  stochastic  perspective.  The  first  stochastic  model  introduces  demographic  variance. 
Subsequent  models  address  population  ceilings,  removals,  and  environmental  variance.  A  particularly 
helpful  feature  of  this  book  is  that  algorithms  are  presented  for  many  of  the  models  presented  in  the  text. 
It  is  therefore  easy  to  follow  the  logic  of  the  risk  calculations  taking  place  in  the  model. 

Quasi  extinction  curves  are  introduced  and  are  shown  to  be  an  effective  means  of  summarizing  the 
models  results.  The  curves  plot  probability  of  various  threshold  population  sizes  over  a  fixed  number  of 
years.  One  of  the  sample  model  runs  reveals  that  over  a  50  year  period  there  is  about  a  40%  chance  the 
population  will  become  extinct. 

In  Chapter  Three  a  nonlinear  model  is  introduced.  Density  dependence  is  one  of  the  primary  features 
added  to  the  model  in  this  chapter.  The  models  address  the  situation  where  high  population  densities  cause 
a  decline  in  population  growth  as  well  as  the  Allee  effect  where  population  density  falls  below  some  critical 
level  and  population  growth  declines.  The  use  of  side  bars  in  this  chapter  and  throughout  the  book  provide 
vivid  and  actual  examples  of  the  effects  being  modeled. 

Several  recent  unstructured  models  are  introduced  in  the  chapter.  The  authors,  in  their  review  of 
these  models,  stress  that  the  choice  of  the  best  model  must  be  based  on  autoecological  information 
concerning  intra  species  interactions,  and  not  just  on  the  statistical  fit  of  a  model  to  observations  of  past 
population  size.  It  is  important  to  have  an  understanding  of  the  population  dynamics  and  life  history  of  a 
species.  Sensitivity  analysis  is  introduced  in  this  chapter  as  a  tool  that  is  recommended  throughout  the 
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remainder  of  the  book.  Testing  the  sensitivity  of  model  results  to  different  values  of  critical  parameters  plays 
a  prominent  role  in  the  risk  assessments  presented. 

The  next  three  chapters  present  structured  population  models.  These  matrix  models  are  designed 
to  capture  the  internal  structure  of  a  population,  using  matrices  to  capture  the  abundances  for  each  of  the 
components  of  the  population.  Though  these  models  yield  substantially  more  detailed  forecasts  they  recjuire 
considerably  more  data  to  parameterize. 

Chapter  Four  presents  age-  and  stage-structured  population  dynamics  models  that  have  been  used 
or  suggested  by  biologists.  Age-structured  models  account  for  the  differences  that  are  important  among 
individuals  related  to  age.  For  example,  differences  in  survival,  mortality,  longevity,  maternity  rate,  age 
distribution  and  abundances,  dispersal,  and  other  demographic  uncertainties  have  all  been  addressed  by  one 
model  or  another  in  the  literature. 

Stage  structure  models  are  suggested  when  the  chances  of  reproducing,  surviving,  dispersing  or 
dying  depend  on  physiological  development  and  an  organism's  status  independent  of  its  age.  The 
mathematics  of  these  models  will  be  difficult  for  many  analysts.  The  Usher  matrix  is  presented  as  a  special 
case  of  the  generalized  Leftovitch  matrix  for  stage-structured  populations.  These  matrices  are  described. 
Like  the  Leslie  matrix  model  developed  for  age-structured  populations,  the  authors  show  how  environmental 
variation  effects  may  be  modeled. 

In  Chapter  Five  spatial  structure  and  metapopulations  dynamics  are  modeled.  Previous  models 
assumed  that  all  individuals,  no  matter  where  they  occur  within  the  spatial  range  of  a  population,  experience 
the  same  changes  in  environment  and  have  the  same  chances  of  surviving  and  reproducing.  Few  biologists 
will  feel  entirely  comfortable  with  such  models.  Apart  from  spatial  variations  in  soil  characteristics, 
hydrology,  and  topography,  more  extreme  events  like  fires,  drought  and  floods  will  have  non-uniform 
effects  on  populations. 

These  models  offer  the  most  promise  for  application  to  habitat  models  now  used  by  the  Corps. 
However,  it  must  be  noted  that  these  are  not  habitat  models,  they  are  dynamic  population  models.  It  just  so 
happens  that  they  address  some  of  the  habitat  issues  that  are  confronted  in  habitat  evaluation  models.  Spatial 
structure  and  metapopulation  dynamics  are  important  for  viability  analysis  and  extinction  risk  assessment 
for  four  main  reasons  according  to  the  authors.  First,  conservation  biology  is  concerned  with  the  protection 
of  species.  Second,  habitat  loss  is  probably  the  major  cause  of  species  extinctions.  Third,  the  existence  of 
multiple  populations  introduces  new  susceptibilities  like  movement  or  isolation  of  populations.  Fourth,  the 
combination  of  nature  reserves  that  gives  a  species  the  highest  chance  of  survival  can  only  be  assessed  by 
analysis  of  a  metapopulation. 

The  two  most  important  aspects  of  metapopulation  dynamics  to  consider  are  the  correlation 
between  environmentally-induced  fluctuations  in  the  vital  rates  of  different  populations,  and  the  rate  of 
dispersal  of  individuals  between  populations.  These  factors  are  frequently  modeled  as  a  function  of  the 
geographic  distance  between  populations.  Occupancy  models  are  presented  to  simplify  the  problem. 
Generalizations  of  the  simple  occupancy  models  are  used  to  model  the  factors  cited  above.  A  new  model 
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presenting  the  dynamics  of  local  populations  as  well  as  environmental  correlations  and  dispersal  is  presented. 
An  example  shows  how  to  determine  the  best  design  of  nature  reserves. 

The  consequences  of  the  loss  of  genetic  diversity  are  presented  in  Chapter  Six.  Adaption  to  change, 
inbreeding  depression,  and  outbreeding  depression  are  addressed  in  models  developed  to  handle  genetic 
variation.  It's  found  that  loss  of  genetic  variability  may  affect  the  ability  of  a  species  to  adapt  to 
environmental  change  and  it  may  reduce  population  average  survivorship  and  fecundities.  The  average  rate 
of  loss  of  alleles  is  modeled  as  a  function  of  population  size. 

"All  but  the  simplest  models  in  this  book  defy  parameterization  because  of  a  lack  of  data. 

This  underlines  the  complexity  of  superficially  simple  questions  such  as  ‘What  increase  in 
risk  results  from  this  management  alternative?’  or  ‘Is  it  better  to  spend  money  on  the 
rehabilitation  of  a  corridor,  or  on  the  extermination  of  a  feral  predator?’  It  also  underlines 
the  scarcity  of  data  essential  to  answer  some  of  the  most  fundamental  questions  facing 
conservation  biologists”  (p.  269). 

The  authors  suggest  that  the  exercise  of  building  a  model  rationalizes  a  problem  and  quantifies  ideas.  These 
results  alone  are  worthwhile,  regardless  of  the  quality  of  the  model  results. 

Some  of  the  more  vexing  uncertainties  of  conservation  biology  are  not  addressed  in  this  book.  For 
example,  it  is  frequently  assumed  that  some  small  population  threshold  can  be  identified  as  critical  to  the 
survival  of  a  population.  This  threshold  is  treated  as  given  throughout  the  text  and  experience  shows  this 
is  one  of  the  fundamental  uncertainties  that  continues  to  vex  analysts.  In  addition,  many  statistics  can  be 
applied  to  conservation  biology  problems.  The  authors  offer  advice  that  would  be  well  heeded  by  Corps 
planners.  When  designing  strategies  to  conserve  or  manage  biological  species,  one  must  choose  which  to 
use  for  decision-making.  A  strategy  that  maximizes  average  population  growth  will  not  necessarily 
maximize  the  time  to  extinction.  The  objectives  of  the  plan  are  critical  to  the  design  of  the  analysis  and  the 
management  strategies. 


99 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 
An  Annotated  Bibliography _ 


A8:  “Ecological  Function  and  Resilience:  Neglected  Criteria  for  Environmental  Impact 

Assessment  and  Ecological  Risk  Analysis.”  By  John  Cairns,  Jr.  and  B.R.  Niederlehner.  The 
Environmental  Professional.  Vol.  15.  No.  1. 1993.  pp.  116-124. 

Applicability 

This  is  an  excellent  article  that  should  be  useful  to  any  environmental  resources  planner.  It  is  an 
extremely  well-written,  balanced  article  that  exhibits  a  clarity  of  argument  rarely  seen.  Additionally,  the 
authors  provide  excellent  citations  to  other  relevant  works  in  the  field.  The  references  alone  are  worth  the 
trouble  of  obtaining  this  article.  We  learned  more  about  ecological  risk  analysis  from  this  article  than  any 
other.  This  paper  will  be  of  particular  interest  to  Corps  personnel  who  will  have  to  develop  risk  analysis 
methodologies  and  guidance.  The  title’s  reference  to  resilience  makes  it  clear  that  this  is  an  important  article 
for  anyone  concerned  with  ecological  restoration.  It  provides  a  framework  for  thinking  about  how  one  might 
go  about  identifying  and  quantifying  risks  to  such  projects.  The  same  framework  would  be  very  useful  for 
traditional  Corps  projects  as  well.  This  is  an  important  article  for  Corps  planners,  one  of  the  very  best 
encountered  in  this  review. 

In  essence,  the  authors  argue  that  using  indicator  species,  a  structural  endpoint  approach  to 
assessment,  may  not  be  the  best  way  to  approach  assessment.  They  argue  effectively  for  the  use  of 
functional  endpoints  and  resiliency  assessments  to  aid  the  decision-making  process. 

Review 


Water  purification,  oxygen  production,  carbon  storage,  climate  regulation,  and  production  of  food, 
wood,  and  medicinal  drugs  are  but  a  few  of  the  functions  of  ecosystems  that  human  society  depends  upon. 
Resilience  refers  to  an  ecosystem’s  ability  for  a  specific  functional  parameter  to  return  to  some 
approximation  of  its  predisturbance  condition.  Judging  the  acceptability  of  risk  requires  knowing  whether 
a  damaged  ecosystem  can  recover  to  an  approximation  of  its  predisturbance  function  and  structure.  In  this 
paper  the  authors  review  the  current  state  of  knowledge  of  ecosystem  function  and  resilience  and  evaluate 
the  incorporation  of  functional  end  points  and  measures  of  ecosystem  resilience  into  environmental  impact 
assessment. 

The  substance  of  the  paper  begins  with  a  critique  of  the  use  of  indicator  species  as  a  measure  of 
environmental  health.  Censuses  of  indicator  species  are  not  particularly  well  thought  of  by  the  authors.  The 
problems  with  the  use  of  indicator  species  include  the  following: 

1)  The  system  is  based  on  a  solid  knowledge  of  the  responses  of  organisms  to  stresses.  There 
are  many  species  for  which  there  are  no  indicators. 

2)  The  responses  of  organisms  to  impact  is  not  uniform  across  stress  type.  The  use  of 
indicator  species  for  one  stress  does  not  accommodate  other  categories  of  stresses 
or  the  cumulative  effects  of  all  the  stresses  that  may  be  present. 
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3)  Environmental  protection  must  assure  organisms  are  robust  and  healthy  not  just 
present. 

4)  The  approach  is  reactive;  it  records,  rather  than  prevents,  damage. 

The  authors  find  that  functional  end  points  are  directly  relevant  to  management  goals  for  environmental 
protection.  They  are  also  species  neutral,  i.e.,  microorganisms  are  as  important  as  macroorganisms. 
Functional  measures  integrate  the  effects  of  stress  on  all  community  members  and  focus  on  properties  that 
are  essential  to  the  sustainability  of  the  ecosystem.  Functional  end  points  may  be  generalized  from  one 
ecosystem  and  stress  type  to  another.  The  article  offers  two  informative  tables  identifying  functional  end 
points  generically  and  for  selected  systems. 

In  general,  Cairns  and  Niederlehner  find  methods  for  determining  ecosystem  function  less  developed 
than  methods  for  determining  ecosystem  structure.  Exceptions  include  sewage  and  forest  productivity. 
Ample  evidence  exists  to  establish  there  is  functional  change  in  stressed  whole  ecosystems.  They  discuss 
the  use  of  regional  reference  habitats  as  baseline  measures  that  would  facilitate  the  use  of  an  index  of  biotic 
integrity  (IBI)  or  other  such  indices.  Their  discussion  of  sensitivity  analysis  of  functional  assessments 
provides  a  well-balanced  review  of  the  literature. 

Of  particular  interest  to  Corps  analysts  will  be  the  discussion  of  resilience.  Not  only  is  it  important 
to  predict  the  magnitude  of  structural  and  functional  effects  of  projects  but  it  is  also  important  to  predict  the 
probable  duration  of  these  effects.  If  changes  are  short-lived  and  small  to  a  redundant  component  of  a 
system  on  a  local  scale,  the  consequence  may  well  be  acceptable.  Changes  that  are  irreversible  or  reversible 
only  over  a  long  period  of  time  that  compromise  unique  resources  may  be  judged  unacceptable.  An 
understanding  of  an  ecosystem’s  ability  to  recover  its  original  structure  and  function  or  some  reasonable 
facsimile  is  an  important  part  of  the  decision  process  concerning  the  risks  of  projects. 

Central  to  environmental  resources  planning  is  the  ability  to  make  predictions  about  environmental 
outcomes  resulting  from  alternative  courses  of  action.  The  authors  argue  that  the  soundness  of  these 
predictions  depends  on  the  availability  of  good  dose-response  models.  After  the  risk  assessment,  the 
acceptability  of  the  risk  is  determined  based  on:  1)  the  magnitude,  time  span,  and  spatial  extent  of  direct 
physical,  chemical,  and  biological  changes  to  the  environment;  2)  the  environmental  end  points  responding 
to  these  changes;  3)  the  magnitude  of  environmental  response;  4)  the  reversibility  of  impairment;  5)  the 
uniqueness  of  the  resources  at  risk;  and  6)  who  benefits  and  by  how  much  versus  who  loses  and  by  how 
much. 


The  article  ends  with  a  discussion  of  some  common  problems  of  risk  assessments.  These  include 
multivariate  stresses  and  responses,  unavoidable  uncertainties,  and  public  perception  of  risks.  On  balance 
this  is  as  well-written  and  thorough  going  a  discussion  as  you  are  likely  to  see  on  the  subject. 


101 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 
An  Annotated  Bibliography _ 


A9:  "Benefits  and  Risk  of  Using  Predators  and  Parasites  for  Controlling  Pests."  By  L.  E. 

Caltagirone  and  C.B.  Huffaker.  Ecological  Bulletin.  No.  31, 1980.  pp.  103-109. 

Applicability 

This  article  is  a  good  introduction  to  the  concept  of  biological  control  of  populations.  It  provides 
just  enough  detail  to  give  the  novice  reader  a  feel  for  the  technique.  The  discussion  of  risk  remains  at  a 
rather  general,  almost  superficial  level.  This  article  is  recommended  for  managers  and  other  non-biologist 
readers.  The  experienced  biologist  may  find  it  elementary. 

Review 


Biological  control  is  defined  here  to  mean  the  regulation  of  populations  by  natural  enemies,  i.e., 
predators,  parasites,  and  pathogens.  Using  biological  controls  to  regulate  systems  of  populations  requires 
an  awareness  of  the  fact  that  prey  and  predator,  parasite  and  host  populations  are  reciprocally-density 
dependent.  These  control  systems  can  be  self-sustaining. 

Biological  control  has  often  been  characterized  by  a  broader  definition  than  offered  here.  The  use 
of  sterile  adults,  plant  resistance,  pheremones  and  other  chemicals  of  biological  origin  are  sometimes 
considered  biological  control  in  other  contexts.  The  fear  that  using  a  natural  enemy  to  eliminate  a  pest,  with 
the  natural  enemy  then  becoming  a  greater  pest  than  the  one  it  was  recruited  to  control,  is  a  common  fear 
that  leads  to  the  question,  "What  are  the  risk  aspects  of  biological  control?" 

Predators  and  parasites  are  used  to  control  pest  populations  in  two  general  situations.  First,  when 
a  species  colonizes  a  new  area  its  population  may  reach  high  levels  because  of  a  lack  of  natural  enemies  in 
the  new  location.  In  this  case,  enemies  have  to  be  recruited  from  native  areas.  Second,  there  could  be  a 
change  in  the  environment  that  changes  the  population  density  of  the  prey  or  parasite  enabling  the  pest 
population  to  grow. 

When  considering  the  safety  of  biological  controls  one  must  distinguish  between  safety  for  humans 
and  safety  for  other  organisms  in  the  environment.  Paper  nest  wasps  are  effective  predators  of  soft  bodied 
insects  but  they  present  a  risk  of  sting  to  humans. 

The  other  possible  risks  are  derived  from  the  impacts  of  predators  and  parasites  on  the  environment. 
When  species  are  transferred  from  one  world  region  to  another  we  must  be  concerned  that  the  predator  or 
parasite  will  eradicate  a  species  or  subsequently  turn  into  a  pest  itself.  Though  the  authors  find  that  few  taxa 
deviate  from  their  special  feeding  habits,  they  recommend  quarantine. 

The  authors  are  unabashed  proponents  of  biological  control.  "The  use  of  predators  and  parasites  for 
pest  control,  when  it  is  the  result  of  a  well  thought  out,  carefully  executed  program  is,  in  our  opinion,  risk¬ 
free.  It  would  appear  that  when  simply  undoing  the  biological  control  is  not  an  option,  quarantining  the 
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predator/parasite  first  is,  in  the  authors'  assessment,  sufficient  to  remove  the  major  risks  of  biological 
control.” 
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AlO:  “Pragmatic  Suggestions  for  Incorporating  Risk  Assessment  Principles  in  EIA  Studies.”  By 
Larry  W.  Canter.  The  Environmental  Professional.  Vol.  15.  No.  1.  1993.  pp.  125-138. 


Applicability 

This  article  will  be  of  particular  interest  to  environmental  resource  planners  specifically  and  water 
resource  planners  generally.  It  is  chock  full  of  interesting  and  useful  lists  and  descriptions  (e.g.,  three  figures 
and  five  tables). 

The  author  starts  from  the  CEQ  and  NEPA  requirements  to  include  a  risk  assessment  (RA)  in  an 
environmental  impact  assessment  (EIA).  Referring  primarily  to  the  EPA  risk  assessment  paradigm  he 
describes  a  number  of  risk  assessment  procedures.  There  are  some  genuinely  useful  ideas  in  these 
procedures.  The  rapid  risk  assessment  described  in  Figure  2,  for  example,  is  an  idea  that  can  be  readily 
adapted  by  Corps  planners  for  a  variety  of  applications. 

Review 


Policies  were  enunciated  in  the  1970s  that  first  required  risk  assessments.  These  risk  assessments 
came  of  age  during  the  1980s  and  continue  to  evolve  in  the  breadth  and  sophistication  of  their  application. 
Canter  uses  the  EPA  model  for  human  health  effects:  hazard  identification,  dose-response  assessment, 
exposure  assessment  and  risk  characterization.  The  introduction  provides  an  overview  of  the  evolution  of 
the  worst  case  scenario  to  a  reasonably  foreseeable  case  and  it  briefly  considers  international  applications 
of  this  type  of  analysis.  He  concludes  that  it  is  timely  to  develop  specific  protocols  and  guidance  that  will 
ensure  the  incorporation  of  risk  assessment  principles  in  the  EIA.  The  potential  benefits  of  including  risk 
assessment  principles  include:  1)  encouragement  of  integrated  thinking  by  the  interdisciplinary  EIA  team; 
2)  the  opportunity  to  focus  attention  on  risk  reduction  activities  such  as  waste  minimization,  pollution 
prevention,  and  mitigating  measures;  and  3)  the  inclusion  of  emphases  on  emergency  response  measures  in 
the  event  of  accidents  and  associated  environmental  perturbations. 

Canter’s  brief  overview  of  the  risk  assessment  process  provides  one  of  the  best  succinct  summaries 
of  the  EPA  risk  assessment  paradigm.  The  four  steps  are  discussed  in  just  enough  detail  to  clearly  define 
them  without  bogging  down  in  detail.  There  is  just  enough  discussion  of  the  difficulties  encountered  at  each 
step  to  leave  the  reader  with  a  general  feeling  of  understanding  for  the  paradigm.  Along  the  way  the  author 
presents  some  very  useful  and  interesting  summaries  of  other  works  in  this  area.  Examples  of  practical 
suggestions  for  incorporating  risk  assessment  in  EIA  studies  are  included  in  the  discussion. 

A  seven-stage  quantitative  risk  assessment  method  is  summarized,  as  is  a  methodology  integrating 
human  health  and  environmental  impacts.  The  EPA’s  on-line  database.  Integrated  Risk  Information  System 
(IRIS),  describing  risk  information  for  over  370  chemicals,  is  described.  The  author  includes  some 
discussion  of  the  literature  that  provide  approaches  for  dealing  with  exposure  assessments,  total  human 
exposure  (THE),  total  exposure  assessment  methodology  (TEAM),  rapid  risk  assessment,  and  the  use  of 
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expert  panels  and  focus  groups.  An  eleven-step  method  for  dealing  with  uncertainty  is  particularly  useful 
and  worthy  of  consideration  should  the  Corps  ever  develop  guidelines  for  dealing  with  this  issue. 

Canter  presents  five  EIS  examples  used  to  address  health  risks.  They  are  of  interest  because  of  the 
different  approaches  used  in  each.  One  employs  a  probabilistic  approach.  Other  approaches  include  a 
qualitative-quantitative  approach  (like  the  rapid  risk  approach  mentioned  earlier),  qualitative  descriptions, 
and  a  relative  risk  approach.  The  range  of  topics  covered  by  existing  risk  assessments  in  EIAs  is  impressive. 
From  drinking  water  to  cannabis  eradication  to  an  impressive  vegetation  management  study  by  the  U.S. 
Forest  Service  that  would  appear  to  warrant  close  study  by  Corp  personnel,  there  is  a  great  range  of  work 
that  has  already  been  done  that  should  not  be  overlooked. 

The  author  concludes  with  some  recommendations.  Two  steps  are  needed  to  enhance  the 
incorporation  of  risk  assessment  into  EISs.  First,  he  advocates  one  or  more  technical  and  policy  related 
conferences  on  the  relationships  between  risk  assessment  and  EIA.  The  target  audiences  are  agency 
personnel,  contractors,  and  private  industry.  Second,  he  advocates  training  courses  for  EIA  practitioners. 
The  most  appropriate  action  he  sees,  however,  is  for  CEQ  regulations  to  be  modified  to  include  risk-related 
requirements  that  address  risk/hazard  assessments  and  the  inclusion  of  contingency  plans. 

There  is  much  of  interest  in  this  paper.  Because  of  its  wide  range  of  topics  there  is  relatively  little 
detail.  As  a  sampler  and  an  entree  to  the  range  of  topics  and  the  literature  it  is  an  excellent  reference  for 
Corps  planners. 


105 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 
An  Annotated  Bibliography _ 


A1 1 :  "A  Risk  Analysis  Methodology  for  Assessing  Natural  Resource  Degradation."  By  R.  Cincotta 
and  F.  Perez-Trejo.  l  and  Degradation  and  Rehabilitation.  Vol.  2,  No.  3,  July-September  1990. 
pp.  191-199. 

Applicability 

In  this  paper  the  authors  define  an  unacceptable  outcome  and  present  a  simple  methodology  for 
quantifying  the  risk  of  that  unacceptable  outcome  as  the  number  of  times  that  outcome  is  observed  over  a 
large  number  of  iterations.  The  method  is  applied  to  a  natural  resource  context  but  is  perfectly  general.  The 
unacceptable  outcome  could  be  the  number  of  times  a  benefit-cost  ratio  falls  below  one  or  it  could  be  the 
number  of  times  a  species  disappears  from  a  project  area  in  a  fifty-year  planning  horizon.  It's  an 
uncomplicated,  well-known  and  neat  idea  that  may  find  applications  in  many  situations  environmental 
resource  planners  face. 

The  contribution  of  this  article  is  one  of  nuance.  By  specifying  thresholds  of  unacceptability, 
simulation  results  can  be  used  to  focus  on  how  often  the  threshold  is  crossed.  This  is  just  a  different  way 
of  looking  at  the  results  of  a  simulation.  Frequently  we  look  at  the  expected  value  and  distribution  of  an 
uncertain  value.  In  this  case  we  simply  want  the  probability  that  a  critical  value  is  equaled  or  exceeded. 

Review 


This  paper  claims  it  "..addresses  the  need  for  evaluative  research  of  resource  strategies,"  a  goal  it 
does  not  meet.  However,  it  provides  a  useful  example  of  risk  analysis  focusing  on  the  notion  of  acceptance 
limits. 


To  achieve  sustainable  production  in  natural  resources  systems  we  must  maintain  the  critical 
components  of  ecosystems  within  acceptable  limits.  What's  "acceptable"  is  a  subjective  judgment  of  the 
resource's  condition.  The  authors  assert  that  societal  notions  of  acceptability  and  unacceptability  can  be 
reduced  to  quantifiable  limits  that  some  groups  strongly  prefer  not  to  cross.  An  acceptance  limit  is  defined 
as  a  function  that  separates  a  domain  of  acceptable  status  from  a  domain  of  unacceptable  status.  Problems 
may  have  several  acceptance  limits  working  at  once.  For  example,  a  population  level  may  be  defined  in 
terms  of  minimum  viable  populations  for  survival,  low  population  levels  that  evoke  poor  hunting  or 
observation  experiences,  or  large  population  levels  that  result  in  intolerable  damages  to  crops.  Three  levels 
of  acceptance  limits  are  identified:  family  level  of  resource  users,  production  system  level,  and  ecosystem 
level. 

The  elements  of  the  basic  risk  analysis  methodology  suggested  by  the  authors  are  as  follows: 

1)  Choose  a  strategy  to  be  applied  to  a  resource  or  resource  use. 

2)  Determine  a  critical  state  variable  that  can  be  used  to  indicate  an  unacceptable 
outcome  for  the  strategy  under  investigation. 

3)  Determine  an  acceptability  limit  for  the  critical  variable. 
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4)  Identify  suitable  decision  rules  for  the  strategy. 

5)  Simulate  the  function  of  the  system. 

6)  Determine  the  risk  that  the  strategy  under  consideration  will  fail  during  the  planning 
period. 

7)  Evaluate  other  parameters  of  risk  and  failure. 

Each  iteration  of  the  simulation  model  is  a  planning  period.  The  risk  of  failure  is  defined  as  the  number  of 
planning  periods  in  the  simulation  in  which  acceptable  limits  are  exceeded,  divided  by  the  total  number  of 
iterations.  Analysts  are  admonished  to  avoid  adding  complexity  to  the  model  that  will  make  it  difficult  to 
communicate  the  risks  to  decision-makers. 

A  hypothetical  example  is  presented.  In  it  a  farmer  must  sell  500  kg  of  product  each  year  to  get  the 
cash  required  for  survival  needs.  Any  extra  is  added  to  storage.  He  needs  a  minimum  of  1,000  kg  in  storage 
for  his  family's  needs.  Fifteen  percent  of  storage  is  lost  to  rot.  The  strategy  under  consideration  is  one  of 
crop  variety  and  cropping  method,  I(t),  that  yields  a  per  hectare  output  as  a  linear  function  of  rainfall  (which 
averages  250  mm  per  year  with  a  standard  deviation  of  100  mm).  A  second  strategy.  Aft),  recommended 
by  the  government,  has  a  non-linear  relationship  to  rainfall.  Aft)  is  better  during  favorable  rainfalls  but 
worse  during  low  rainfalls.  The  model  is  further  complicated  by  other  constraints  on  the  government 
strategy. 

Using  a  30-year  planning  period  and  random  rainfall  patterns,  simulations  for  the  two  strategies,  Ift) 
and  Aft),  are  run.  Unacceptable  outcomes  are  those  that  fail  to  produce  enough  food.  The  parameters 
reported  are:  risk  probability  (number  of  planning  periods  that  had  one  or  more  failures  divided  by  the  total 
number  of  planning  periods  (a  planning  period  is  a  single  iteration  of  the  simulation  model));  failure  rate 
(average  number  of  unacceptable  states  per  planning  period  with  one  or  more  failures);  range  of  failures 
(maximum  number  of  failures  per  planning  period  minus  minimum  number  per  planning  period);  and  the 
median  value  of  a  failure  (median  production  level  in  kg  for  a  failed  production  level).  The  analysis  showed 
the  Ift)  strategy  to  be  the  better  of  the  two. 

The  authors  conclude  that  risk  analysis  is  important  as  a  learning  tool.  The  exercise  of  identifying 
critical  variables  and  acceptance  limits  raises  vital  issues  of  which  analysts  must  be  aware.  This  too  is  a 
point  well-taken  by  Corps  analysts  embarking  on  environmental  resource  risk  analysis. 
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A12:  "Impact  of  Discharges  as  a  Basis  for  Control  Measures".  By  R.A.  Conway.  Water  Science  and 
Technology.  Vol.  14,  pp.  95-109, 1982. 

Applicability 

The  title  and  list  of  key  words  for  this  article  sounded  promising.  The  article  turned  out  to  be 
focused  narrowly  on  water  quality  and  it  was  somewhat  dated  for  the  purposes  of  this  review.  It  contains 
terminology  that  would  now  be  considered  incorrect  in  usage,  though  in  fairness  to  the  author  these  were 
terms  that  undoubtedly  were  struggling  for  an  identity  in  1982.  It  is  of  limited  utility  today. 

Review 


The  purpose  of  this  article  was  to  make  a  case  for  additional  studies  to  assess  the  impact  of 
discharges  upon  the  aquatic  environment  in  order  to  allow  water  quality  managers  to  tailor  the  control 
measures  used  to  the  Impacts  observed.  Observing  advances  in  the  environmental  risk  assessment  field  at 
the  time  the  article  was  written,  the  author  proposes  that  the  risk  assessment  methodologies  might  be 
adapted  for  use  by  water  quality  managers.  The  author  cites  several  studies  that  seem  to  argue  that  at  times 
there  is  little  environmental  benefit  achieved  by  the  large  expenditures  on  capital  and  energy  for  water 
treatment. 

Conway  identifies  the  stochastic-statistical  approach,  the  model  ecosystem,  the  deterministic 
approach,  and  the  baseline  approach  as  the  four  approaches  to  environmental  risk  analysis.  Though  this 
classification  might  be  challenged  today,  he  chose  to  use  the  deterministic  approach  to  examining  the  fate 
and  effects  of  chemicals  in  the  environment.  The  deterministic  approach  might  today  be  considered  a  form 
of  sensitivity  analysis  that  relies  on  identification  and  parametric  variation  of  key  variables  in  the  model. 

A  hazard  assessment  model  showing  the  relationship  between  chemical  fate  and  effects  is  presented. 
Candidate  tests  for  screening  ecological  impacts  of  new  effluents  are  identified  for  chemical  fate  and 
ecological  effects.  Data  required  for  modeling  environmental  input  (entry),  transport  (mobility),  and 
transformation  (persistence)  are  used  to  estimate  an  effective  environmental  concentration  of  the  effluent 
or  chemical  agent.  The  resulting  concentration  is  compared  to  known  toxicity  information  to  determine  the 
significance  of  impacts.  Selected  mechanisms  affecting  the  fate  of  chemicals  for  the  input,  transport,  and 
transformation  functions  are  listed.  In  Table  4,  key  environmental  properties  of  physical,  chemical  and 
biological  transport  are  identified. 

The  article  identifies  key  components  of  a  deterministic  approach  model  and  proceeds  to  the  point 
of  listing  factors  and  variables  to  consider.  The  article  stops  short  of  offering  any  analysis  and  is  of  limited 
utility  today. 
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A13:  “What  is  Ecological  Economics?”  By  Robert  Costanza.  Ecological  Economics.  Vol.  1.  No.  1. 
February  1989.  pp.  1-7. 

Applicability 

The  following  article  launched  a  new  journal.  Ecological  Economics.  It  was  selected  not  so  much 
because  of  its  direct  relevance  to  risk  and  uncertainty  analysis  in  environmental  resource  planning  contexts, 
but  because  we  did  not  know  what  ecological  economics  was.  It  seemed  to  be  a  new  area  of  concentration 
that  could  be  of  interest  to  analysts  working  in  both  the  ecological  and  economic  fields.  This  would  include 
many  Corps  analysts.  The  article  is  a  simple  introduction  to  the  concentration  and  the  journal,  a  journal  that 
looks  like  it  will  be  well  worth  monitoring  by  environmental  planners  and  risk  analysts. 

Review 


“Ecological  Economics  is  a  new  field  formed  at  the  intersection  of  two  older  ones”  (p.  5).  It 
addresses  the  relationships  between  ecosystems  and  economic  systems  in  the  broadest  sense.  It  includes 
neoclassical  environmental  economics  and  ecological  impact  studies  as  subsets,  but  it  is  formed  to  encourage 
new  ways  of  thinking  about  the  linkages  between  ecological  and  economic  systems.  It  implies  a  holistic 
view  of  the  problem  of  studying  and  managing  our  world. 

The  author  notes  that  the  most  insidious  form  of  ignorance  is  misplaced  certainty.  Current  economic 
paradigms  rely  heavily  on  the  assumption  of  continuing  growth.  An  economy’s  health  is  often  measured  by 
a  stable  and  high  rate  of  growth  of  its  gross  domestic  product  (GDP).  Costanza  differentiates  between  two 
world  views  that  differ  in  strikingly  important  ways.  First,  there  is  the  technological  optimist.  According 
to  this  paradigm,  energy  and  resource  limits  to  growth  will  be  overcome  by  new  technology.  The 
technological  pessimist,  on  the  other  hand,  assumes  technology  will  not  be  able  to  circumvent  fundamental 
energy  and  resource  constraints  and  that  economic  growth  must  eventually  stop. 

Ecologists  have  frequently  been  among  the  latter  group,  largely  because  in  natural  systems  resource 
constraints  always  limit  growth.  Economists  tend  to  fall  into  the  prior  group.  Costanza  argues  that  there  are 
vast  differences  in  the  specific  public  policies  we  should  pursue,  depending  on  whether  the  optimists  or 
pessimists  are  right.  Optimists  argue  that  unless  we  believe  the  optimistic  future  is  possible  and  behave 
accordingly,  it  won’t  come  to  be.  Pessimists  believe  that  such  pursuit  of  growth  will  bring  the  inevitable 
decline  on  more  quickly.  To  sustain  our  system  we  should  begin  to  conserve  resources  now.  These  are  two 
fundamentally  opposing  views  of  the  future.  Each  is  compelling  in  the  simplicity  of  its  argument,  each  with 
profound  implications  for  life  on  earth. 

The  conflicts  in  these  two  world  views  can  be  seen  in  the  work  of  the  Corps.  It’s  more  complicated 
than  an  NED  versus  EQ  debate,  but  every  Corps  analyst  has  seen  intractable  situations  in  which  local 
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interest  and  resource  agencies  have  been  unable  to  communicate  effectively.  We  suspect  that  is  often 
because  of  the  fundamental  differences  Costanza  describes  in  this  article. 

Costanza  appears  to  side  with  the  pessimists  as  he  concludes  too  little  attention  is  being  paid  to 
policies  based  on  technologically  pessimistic  assumptions.  Pursuing  these  policies  or  having  them  be  one 
of  the  alternatives  should  be  part  of  our  policy  dialog.  The  true  energy  and  environmental  state  of  the  world 
may  be  the  greatest  uncertainty  we  face  in  environmental  resource  planning,  a  point  we  are  well-advised  to 
never  forget.  Issues  of  sustainability  are  issues  about  limits.  Issues  about  limits  are  issues  about  uncertainty. 
If  we  are  uncertain  about  limits,  the  prudent  course  is  to  assume  they  exist.  Or,  we  might  add,  we  should 
develop  alternatives  that  assume  they  exist. 


no 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography 


A14:  "Prospect  Risk  Analysis  Applied  to  Ground-Water  Reservoir  Evaluation".  By  George  H. 
Davis.  Ground  Water.  Vol.  20,  No.  6,  pp.  657-662,  November-December  1982. 

Applicability 

One  of  the  advantages  of  the  earlier  risk  literature  is  that  it  frequently  meets  the  needs  of  analysts 
just  being  introduced  to  the  topic  or  just  beginning  to  apply  it  to  new  fields.  As  risk  analysis  is  extended  into 
new  areas  it  will  have  to  start  modestly.  This  easy-to-read  paper  provides  a  simple  example  of  how  risk 
analysis  can  be  applied  to  the  quantification  of  uncertain  resources.  It  provides  helpful  lists  of  questions, 
steps  in  estimating  quantities  of  uncertain  water  resources  and  an  easy-to-follow  example  presenting  the 
results  in  a  convenient  cumulative  distribution  function.  It  relies  on  a  simulation  using  the  Monte  Carlo 
process.  Some  of  the  language  used  by  the  author  is  stilted  and  is  no  longer  in  general  use  by  risk  analysts, 
if  it  ever  was.  It  is  recommended  for  readers  relatively  new  to  risk  analysis. 

Review 


This  paper  applies  probabilistic  methods  to  the  problem  of  estimating  the  storage  capacity  and  hence 
initial  yields  of  ground-water  reservoirs.  The  process  used  in  the  past,  for  lack  of  better  methods,  has  been 
to  use  all  the  available  data  in  basin-wide  estimates  without  differentiating  the  varying  certainty  we  have  in 
our  data. 

Developing  water  resources  is  a  risky  undertaking.  According  to  the  author,  the  primary  questions 
to  answer  in  a  risk  analysis  of  this  type  are  identified  as:  1)  How  large  is  the  resource?  2)  Where  is  it 
located?  3)  In  what  size  deposits  does  it  occur?  4)  How  certain  are  we  of  our  opinions?  5)  How  long  will  the 
resource  last  under  various  operating  alternatives? 

The  method  of  analysis  is  divided  into  two  parts.  The  first  is  estimating  volume  factors,  including 
area,  aquifer  thickness,  and  storativity.  The  second  part  is  estimating  the  risk  factors  that  apply  to  these 
critical  variables.  This  technique  is  a  generally  applied  and  useful  one.  Identifying  critical  variables  and 
quantifying  the  uncertainty  attending  these  variables  would  be  a  useful  first  step  for  environmental  resource 
risk  analysts  as  well. 

The  procedure  used  in  this  particular  study  can  be  summarized  by  the  following  seven  steps.  First, 
assemble  all  available  hydrogeologic  data  on  the  ground-water  reservoir.  Second,  summarize  the 
information.  Third,  quantify  the  data  to  the  extent  feasible.  Fourth,  convene  a  panel  of  hydrogeologic 
experts  who  individually  review  the  data.  Fifth,  each  expert  provides  a  subjective  estimate  of  storage 
capacity.  Sixth,  results  are  reviewed  by  the  group  of  experts.  Seventh,  the  subjective  estimates  of  the 
experts  are  used  in  a  Monte  Carlo  simulation  to  derive  probabilistic  estimates  of  the  reservoir  under 
consideration. 
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This  technique  of  using  subjective  probability  assessments  to  quantify  uncertainties  in  a  qualitative 
manner  is  likely  to  be  an  important  tool  to  environmental  resource  planners.  The  description  of  the  technique 
and  presentation  of  the  results  are  now  relatively  routine.  Distinctions  between  "risked"  and  "unrisked" 
values  are  not  mainstream  distinctions,  but  this  is  a  minor  distraction.  The  author  recaps  the  value  of  this 
article  best  in  his  own  words  in  his  summary.  The  method  presented  by  Davis  offers  advantages  over  the 
former  determinate  method  because  it: 

1 )  Incorporates  uncertainty  into  the  resource  estimate. 

2)  Displays  the  possible  range  of  estimates,  and  accommodates  widely-differing 
views. 

3)  Breaks  the  assessment  procedure  into  component  parts  that  can  be  critically 
examined. 

4)  Provides  ease  of  revision  through  updating  of  components. 

5)  Provides  a  systematic  approach  to  a  topic  that  has  not  been  handled  in  a  systematic 
way  in  the  past. 
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A15:  “Risk  Computation  for  Environmental  Restoration  Activities.”  By  J.  G.  Droppo,  Jr.,  J.W. 
Buck,  D.L.  Strenge  and  B.L.  Hoopes.  Journal  of  Hazardous  Materials.  Vol.  35. 1993.  pp.  341« 
352. 

Applicability 


Environmental  restoration  has  become  one  of  the  Corps  of  Engineers’  priority  missions.  This  article 
presents  one  example,  site  cleanup,  of  what  environmental  restoration  means  to  the  Department  of  Energy. 
The  doe’s  emphasis  on  site  cleanup  gives  them  a  more  narrow  focus  than  the  Corps’  restoration  program 
will  have. 

This  paper  is  of  interest  to  Corps  analysts  for  a  number  of  reasons.  First,  it  is  useful  for  any  insight 
it  provides  on  the  issue  of  risk  analysis  for  restoration.  Second,  it  has  obvious  implications  for  any  future 
Superfund  activities  of  the  Corps.  Third,  it  provides  an  introduction  to  the  notion  of  using  a  risk  screening 
tool  for  preliminary  decisions  about  relative  risks.  Any  one  of  these  contributions  makes  the  article  worth 
reading.  To  have  all  three  together  makes  it  a  very  useful  article  for  any  analysts  seeking  better 
understanding  of  risk  analysis  for  environmental  restoration.  Preliminary  interest  in  possibly  developing  a 
single  risk  analysis  model  for  selected  environmental  resource  planning  activities  would  be  well  served  by 
examining  DOE’s  experience  with  a  single  model  approach  described  in  this  paper. 

Review 


Environmental  restoration  activities  covered  by  the  Comprehensive  Environmental  Response, 
Compensation  and  Liability  Act  (CERCLA)  must  address  environmental  and  public  health  risks.  The 
remedial  process  begins  with  problem  identification  and  leads  to  final  site  cleanup.  At  each  stage  of  this  site 
cleanup  process  risk  evaluations  are  required.  In  the  early  stages  uncertainty  is  greatest,  data  are  weakest, 
and  risk  computation  requirements  are  less  demanding  than  they  become  as  the  process  progresses. 

A  number  of  computer  models  are  used  to  estimate  risks  related  to  environmental  restoration  of  sites 
contaminated  with  hazardous  wastes.  The  authors,  from  Battelle  Pacific  Northwest  Laboratory,  describe  the 
Multimedia  Environmental  Pollutant  Assessment  System  (MEPAS),  one  such  model  used  during  the  early 
stages  of  analysis  to  screen  the  order  of  magnitude  of  risks.  MEPAS  enables  the  user  to  establish  risk-based 
priorities  from  among  a  large  number  of  potential  problems. 

The  model  uses  readily  available  information  on  contaminant  characteristics,  release  rates,  and  other 
factors,  as  model  inputs.  Standard  computational  techniques  are  used  to  model  transport  pathways  and 
exposure  pathways.  Outputs  are  identified  in  terms  of  human  health  impacts.  The  unique  contribution  of 
MEPAS  is  that  it  has  integrated  components  of  many  separate  models  into  a  single  system.  It  is  used  to 
identify  problems  that  merit  additional  resources  for  further  study.  Version  3.0,  developed  for  the  personal 
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computer,  is  described  briefly  in  the  paper.  MEPAS  has  been  applied  to  both  multiple  site  and  site-specific 
studies. 


An  example  of  a  risk-based  ranking  of  wastes  for  a  single  site  is  provided  in  the  paper.  MEPAS  was 
used  to  establish  the  relative  importance  of  radioactive  and  nonradioactive  constituents  potentially  stored 
in  underground  single-shell  tanks  at  DOE’s  Hanford  Site  in  Washington.  The  potential  human  health 
impacts  that  result  from  groundwater  transport  of  different  chemical  constituents  were  analyzed.  The 
sensitivity  of  the  hazard  ranking  to  recharge  and  transport  rates  was  also  investigated.  The  results  show 
many  orders  of  magnitude  in  differences  in  risk-based  rankings  of  constituents.  MEPAS  has  apparently 
proved  to  be  a  valuable  scoping  tool.  From  a  list  of  about  70  constituents,  five  radionuclides,  headed  by 
carbon- 14,  and  eight  noncarcinogenic  chemicals,  headed  by  cyanide  ion,  were  identified  as  most  threatening. 

The  notion  of  a  single  integrated  model  that  facilitates  initial  risk  screening  of  problems/management 
measures  is  an  idea  that  would  likely  appeal  to  Corps  analysts.  Whether  such  a  model  is  feasible  or  even 
desirable  is  a  long  way  from  being  determined.  An  important  first  step  in  considering  the  development  of 
such  a  tool  would  be  to  find  out  more  about  the  MEPAS  model  and  how  well  it  has  served  the  DOE  decision¬ 
making  process. 
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A16:  "Application  of  Environmental  Risk  Analysis  to  Goundwater  Protection".  By  M.O.  Ettala. 
Water  Science  Technolo^.  Vol.  20,  No.  3,  pp.  87-93, 1988. 

Applicability 

This  paper  presents  a  technique  for  dealing  with  risks  in  a  subjective  or  qualitative  manner  that  may 
be  very  useful  to  environmental  resource  planners  who  are  struggling  with  how  to  begin  to  apply  risk 
analysis  in  this  new  arena.  It  offers  lists  of  questions  and  information  sources  that  might  be  helpful,  in  a 
suggestive  fashion,  to  planners  trying  to  get  started  with  environmental  resource  risk  analysis.  There  is  really 
little  of  substantive  use  in  the  paper,  and  is  woefully  short  on  the  details  of  the  risk  analysis  and  the 
techniques  the  paper  purports  to  be  examining. 

This  is  an  easy-to-read  paper  for  anyone  interested  in  exploring  simple,  beginner  methods  for 
estimating  risks  in  a  new  subject  area.  For  this  purpose  it  is  recommended.  Those  seeking  help  in 
groundwater  risk  analysis  will  be  disappointed. 

Review 


Industrial  hazards  are  divided  into  risks  to  property,  persons,  know-how  and  operations.  One  kind 
of  operations  risk  is  environmental  impairment  liability.  The  increase  in  such  risks  has  given  rise  to  the  need 
for  prevention  of  environmental  hazards.  The  aim  of  this  paper  was  to  present  a  method  of  analysis  that  will 
make  it  possible  to  identify  and  assess  the  risks  to  groundwater  involved  in  industrial  operations  and  to 
present  measures  for  controlling  them.  In  addition,  the  author  examined  the  applicability  of  fault  trees, 
action  error,  hazard  and  operability  study  (HAZOP),  and  management  oversight  and  risk  tree  (MORT). 

Four  risk  analyses  were  conducted  over  the  period  1985-87  in  the  chemical,  forest  and  foodstuff 
industries.  A  risk  model  consisting  of  the  following  stages  was  used:  1)  identification  of  risks;  2) 
assessment  of  magnitude  of  risks;  and,  3)  corrective  measures  proposed.  The  studies  revealed  that  the  causes 
and  consequences  of  risk  may  be  so  complex  that  a  working  group  is  required  to  address  the  risk  analysis. 

The  author  suggests  the  working  group  should  consist  of  four  to  eight  experts.  The  working  group 
should  be  an  interdisciplinary  team  that  comprises  experts  in  risk  analysis,  the  relevant  industrial  processes, 
personnel  responsible  for  matters  relating  to  environmental  protection,  and  management. 

Lengthy  lists  of  information  sources  and  interview  questions  that  may  be  relevant  for  groundwater 
protection  are  included  in  the  article.  Ettala  found  that  it  was  not  easy  to  quantify  the  consequences  of 
environmental  risks  in  money  terms  and  devised  a  risk  scale.  The  consequences  ranged  from  1  =  negligible 
to  5  =  very  serious.  The  probabilities  of  these  consequences  were  classified  from  1  =  very  improbable  to  5 
=  very  probable. 
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The  results  of  the  analysis  are  presented  in  a  qualitative  form.  For  example,  one  source  of  risk  or 
cause  of  damage  was  identified  as  "continuous  overflow  from  production  lines".  The  consequences  of  this 
hazard  are  "corrosion  of  floors,  floor  drains  and  sewers,  pollution  of  groundwater".  These  consequences 
were  ranked  a  four,  with  five  a  maximum  damage.  The  likelihood  of  this  hazard  is  a  five,  very  probable. 
Potential  measures  to  control  the  risks  include  "elimination  of  overflow,  more  careful  selection  of  materials, 
training".  The  presentation  of  these  and  other  results  in  Table  1  of  the  paper  present  a  subjective/qualitative 
approach  to  risk  analysis  that  may  be  useful  to  environmental  resource  planners  while  awaiting  development 
of  more  analytical  techniques. 
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A17:  “Valuing  Environmental  Resources  Under  Alternative  Management  Regimes.”  By  A.  Myrick 

Freeman,  III.  Ecological  Economics.  Vol.  3.  No.  3.  September,  1991.  pp.  247-256. 

Applicability 

Much  of  our  understanding  and  knowledge  of  the  world  we  live  in  is  uncertain  or  risky.  Increasingly 
more  of  this  uncertainty  is  turning  up  in  books  and  articles  about  risk  and  uncertainty  analysis.  There  is  still 
a  great  deal  of  the  essential  uncertainty  that  is  embedded  in  all  that  we  do.  Uncertainty  is  so  prevalent  that 
it  is  often  accepted  without  particular  notice.  This  article  is  an  excellent  example  of  how  uncertainty  is 
routinely  analyzed  without  any  explicit  recognition  of  the  fact  that  we  are  dealing  with  fundamental 
uncertainties. 

This  is  an  economics  paper  that  deals  with  two  separate  management  alternatives  for  the  blue  crab 
fishery.  One  is  perfect  regulation  (i.e.,  efficient  in  an  economic  sense),  while  the  other  is  open  access 
fishery.  The  analysis  makes  extensive  use  of  assumptions  in  its  modeling  and  presents  an  explicit  sensitivity 
analysis.  At  no  time,  however,  do  risk  or  uncertainty  appear  as  explicit  themes  in  this  paper.  We  have 
observed  during  the  course  of  this  literature  review  that  this  is  a  common  occurrence.  Hence,  the  inquisitive 
analysts  should  never  overlook  a  promising  title  simply  because  the  words  risk  or  uncertainty  appear 
nowhere  in  the  title  or  the  list  of  key  words.  There  is  a  great  deal  of  interest  to  risk  analysts  going  on  out 
there  and  ideas  can  be  found  in  seemingly  unlikely  places  from  time-to-time. 

A  handy  idea  here  is  that  environmental  resources  produced  or  protected  by  a  project  may  not  have 
an  observable  market  value.  However,  if  the  resource  is  an  input  to  the  production  of  something  that  does 
have  an  observable  market  value,  the  value  can  be  estimated  from  that  other  market.  Freeman  uses  changes 
in  the  value  of  blue  crabs  to  estimate  the  value  of  changes  in  the  amount  of  wetlands.  It’s  a  well  known 
technique  for  economists. 

Review 


Freeman  uses  the  data  from  a  1987  Ellis  and  Fisher  article  (EF)  to  calculate  the  economic  value  of 
different  resource  management  alternatives.  EF  describe  the  theoretic  welfare  economics  basis  for 
estimating  the  value  of  environmental  resources  that  are  inputs  in  the  production  of  marketable  commodities 
such  as  fish.  Specifically,  they  looked  at  the  value  of  wetlands  as  an  input  in  the  production  of  the  Gulf 
Coast  blue  fin  crab  fishery.  Economic  value  in  this  article  is  described  as  the  sum  of  producer  and  consumer 
surpluses  and  changes  in  these  surpluses  associated  with  different  amounts  of  wetland  acreage. 

Freeman  looks  at  two  different  management  regimes  for  the  fishery.  Because  the  EF  paper  equated 
price  and  marginal  cost.  Freeman  concludes  this  is  consistent  with  a  perfectly  competitive  market  structure 
(open  access  fishery),  or  that  the  resource  is  a  common  property  resource  with  optimal  regulation  of  effort 
and  catch.  Theory  suggests  that  in  the  long-run  under  perfect  competition  there  will  be  no  producer  surplus. 
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The  value  of  the  resource  is  expected  to  vary  under  the  two  regimes.  This  paper  shows  the  value  of  the 
resource  could  be  lower  under  optimal  management  than  open  access,  a  result  that  some  find  counter 
intuitive. 

The  model  basically  allows  the  author  to  generate  supply  and  demand  curves  and  estimate  areas 
under  them.  To  do  this,  an  iso-elastic  demand  function  and  a  Cobb-Douglas  production  function  are 
assumed.  The  academic’s  ability  to  assume  functional  forms  gets  the  author  over  a  hurdle  that  Corps 
analysts  cannot  often  leap.  While  the  theory  and  methods  are  often  of  most  interest  in  the  journals,  in  the 
planner’s  world  the  magnitudes  of  numbers,  the  data  they  came  from,  and  the  techniques  used  are  subjected 
to  more  rigorous  review. 

The  analysis  starts  from  an  assumed  base  of  25,000  acres  of  wetland  and  looks  at  changes  in  the 
value  of  the  crab  resource  to  measure  the  value  of  wetlands  for  increases  to  100,000,  300,000,  and  500,000 
acres  of  wetlands.  The  analysis  is  a  nice  example  of  sensitivity  analysis  or  what  the  spreadsheets  now  call 
a  data  table.  The  values  of  wetlands  are  estimated  for  three  different  amounts  of  wetlands,  at  four  different 
demand  elasticities,  for  two  management  regimes  (regulation  and  open  access). 

The  results  show  that  different  management  regimes  are  optimal  depending  on  demand  elasticities. 
This  article  can  be  very  helpful  to  Corps  analysts  in  showing  a  way  to  approach  the  problem  of  valuing 
something  that  does  not  appear  in  the  market.  Wetlands  are  valued  by  observing  their  effects  on  the  crab 
market.  The  value  of  wetlands  is  shown  to  change  with  the  manner  in  which  the  fishery  is  managed.  The 
good  news  is  that  these  are  ideas  that  can  be  applied  to  many  situations.  The  bad  news  is  that  the  results 
were  contingent  upon  the  author’s  assumptions  about  apparently  unobserved  demand  and  production 
functions.  Results  were  very  sensitive  to  elasticity  of  demand  which  was  also  unknown.  With  more  data 
this  technique  shows  considerable  promise. 
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A18:  “Public  Policy  and  Environmental  Risk:  Political  Theory,  Human  Agency,  and  the  Imprisoned 

Rider.”  By  John  Martin  Gillroy.  Environmental  Ethics.  Vol.  14.  Fall  1992.  pp.  217-237. 

Applicability 

This  is  a  difficult  article  for  those,  like  this  reviewer,  unfamiliar  with  the  literature  on  human  and 
moral  agency.  In  fairness  to  the  author,  we  may  not  have  fully  appreciated  his  finest  points.  However,  the 
author,  a  political  scientist,  suggests  that  using  economic  welfare  analysis  as  the  basis  for  resolving 
environmental  risk  situations  is  inadequate.  The  author  finds  that  the  economist’s  rational  choice  model  fails 
to  account  for  human  agency. 

There  are  many  within  and  without  the  Corps  who  would  agree  that  an  economic  analysis  of  risky 
situations  is  by  itself  inadequate.  Few  economists  would  argue  with  this  point.  In  this  point  the  author  has 
much  company.  Where  many  will  part  company  with  the  author  is  over  his  impression  that  risk-benefit 
analysis  is  the  only  criteria  that  matters.  That  is  surely  not  the  planning  context  created  by  the  Principles  and 
Guidelines  used  by  the  Corps.  The  author  seems  to  miss  the  obvious  point  that  economics  offers  but  one 
approach  to  assess  and  evaluate  risky  situations. 

The  author’s  assault  on  the  economic  paradigm  for  risk  analysis  is  weakened  by  the  fact  that  he 
makes  several  errors  in  interpreting  economic  theory.  The  critique  of  economics  is  well-intentioned  but 
something  less  than  effectively  executed.  This  article  will  be  of  interest  to  those  who  enjoy  a  more 
philosophic  approach  to  the  problem  of  environmental  risk.  Once  again,  the  emphasis  is  on  the  human  health 
dimension  of  environmental  risk. 

Review 


Gillroy  begins  with  an  overview  of  risk  and  the  economic  point  of  view.  He  finds  that  the  economic 
criteria  lack  the  proper  moral  grounds  to  serve  as  guideposts  to  the  analysis  of  policy  options.  The 
economist’s  assumptions  “..do  not  adequately  address  the  strategic  and  moral  reality  of  environmental  risk 
dilemmas,  and  therefore  prescribe  policy  that  does  not  address  what  is  fundamentally  at  stake  in  these 
decisions.”  Leaving  the  details  of  his  arguments  to  those  motivated  to  read  the  article,  we  found  the  review 
was  based  on  a  superficial  and  incomplete  understanding  of  the  economics  paradigm  generally  and  its 
application  to  risky  situations  specifically. 

Defining  environmental  quality  as  a  public  good,  Gillroy  begins  to  build  his  argument  for  a  different 
set  of  guideposts.  On  the  way  to  developing  his  notion  of  an  “imprisoned  rider”  he  translates  the  well  known 
prisoners’  dilemma  to  the  polluter’s  dilemma.  This  problem  has  been  used  to  show  that  when  individuals 
are  allowed  to  act  in  their  own  self-interest  their  strategic  decisions  will  lead  society  to  a  sub-optimal 
outcome.  If  individuals  could  somehow  be  required  to  or  coerced  into  cooperating,  everyone  would  end  up 
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better  off.  In  the  case  of  pollution,  Gillroy  shows  that  self  interest  in  the  use  of  a  public  good  will  lead  to 
more  pollution  than  would  result  if  we  cooperated.  These  results  are  well  known  in  the  literature. 

Gillroy’s  contribution  is  that  he  develops  the  idea  of  a  critical  mass  of  people  who  must  be  required 
to  go  along  with  the  collective  standard  for  environmental  quality.  Even  in  this  development  Gillroy  seems 
to  be  oblivious  to  the  fact  that  his  arguments  will  hold  for  only  a  very  narrow,  and  generally  uninteresting, 
set  of  circumstances. 

The  author  provides  a  compelling  argument  for  government  intervention  in  the  protection  of 
environmental  quality.  Government  must  regulate  to  prevent  a  catastrophe  from  bringing  about  collective 
and  irreversible  harm.  He  warns  that  free-riding  on  the  provision  of  environmental  quality  can  lead  to  such 
a  catastrophe  because  it  doesn’t  take  many  defectors  to  do  serious  environmental  damage  if  they  are  emitting 
particularly  noxious  pollutants. 

Building  on  his  polluters’  dilemma,  Gillroy  argues  that  there  is  a  significant  asymmetry  of  power 
in  the  strategic  game  over  environmental  quality.  The  players  in  a  simple  two  actor  game  can  be  seen  as  a 
group  of  individuals  who  have  no  power  to  pollute  and,  therefore  impose  no  risks  on  themselves  or  the  other 
actor;  and,  the  other  player  who  is  a  polluter.  The  polluter  has  an  obvious  incentive  to  pollute  in  order  to 
hold  down  costs.  Thus,  one  group  has  risks  imposed  upon  them  with  no  apparent  gain.  When  there  is  no 
regulation  in  such  a  situation,  the  optimal  strategy  will  be  for  the  polluter  to  pollute,  an  even  less  desirable 
outcome  than  when  we  compare  two  polluting  players. 

This  exploitation  of  unorganized  individuals  can  proceed  without  their  direct  knowledge.  It  leads 
to  the  worst  payoff  for  the  citizens  and  the  best  for  the  polluter.  Gillroy  calls  the  individual  in  this  situation 
an  imprisoned  rider.  He  can  only  be  freed  through  some  coercive  force  that  will  define  and  protect  public 
interest.  This  means  risks  must  be  distributed  and  managed  fairly. 

Gillroy  rejects  what  he  describes  as  the  economist’s  reliance  on  preference  theory  as  the  basis  for 
allocating  risks.  He  substitutes  agency  of  the  individual  as  the  standard  for  policy  making.  His  definition 
of  agency  did  not  provide  an  apparent  conflict  with  economic  theory.  In  fact,  it  provided  no  distinction  from 
that  theory  that  this  reviewer  could  ascertain.  That  is  undoubtedly  the  reviewer’s  shortcoming.  Agency  is 
a  concept  that  gets  at  the  individual’s  vital  and  intrinsic  values  that  transcend  mere  preferences.  There 
follows  a  somewhat  protracted  argument  about  the  policy  imperatives  of  human  agency. 

“The  imposition  of  an  environmental  risk  is  an  act  of  disrespect  on  the  part  of  one  actor  who 
increases  his  or  her  personal  welfare  at  the  expense  of  others’  vital  capacity  and  intrinsic 
value.  The  essential  component  of  environmental  risk  is  its  potential  to  harm  the  most  basic 
capacities  of  the  individual”  (p.  233). 

The  author  draws  a  distinction  between  risks  that  can  be  chosen  and  those  that  are  imposed.  The  former  are 
a  product  of  human  agency,  the  latter  are  the  lot  of  the  imprisoned  rider.  It  is,  he  concludes,  the  role  of 
government  to  eschew  the  use  of  subjective  welfare  calculations  and  free  the  imprisoned  rider  through  ex 
ante  regulation  of  environmental  risks.  This  conclusion  will  find  many  supporters. 
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A19:  "Risk  Analysis,  Fuzzy  Logic  and  River  Basin  Management."  By  P.W.  Jowitt.  Water  Science 
and  Technology:  A  Journal  of  the  International  Association  on  Water  Pollution  Research.  Vol. 
16,  No.  5-7, 1984.  pp.  579-585 

Applicability 

This  article  was  chosen  to  be  included  in  this  review  because  of  its  title.  Its  central  theme  is  that  the 
robustness  of  the  model  is  more  important  than  its  sophistication.  That  seems  a  very  worthy  idea  for 
environmental  resource  planners  to  bear  in  mind.  We  don’t  need  large  complex  models  as  much  as  we  need 
models  that  are  capable  of  providing  useful  results.  This  article  is  of  rather  limited  applicability. 

Review 


Water  pollution  control  measures  are  designed  to  reduce  the  mean  level  of  some  pollutant  over  time. 
Considering  the  effects  of  a  temporary  operational  failure  resulting  in  a  solids  overflow,  Jowitt  poses  several 
dilemmas.  He  notes  that  most  pollution  incidents  are  not  well-defined.  Typically,  both  the  river  and  the 
treatment  plant  behave  in  a  “noisy”  fashion.  The  relationship  between  upstream  and  downstream  conditions 
is  highly  corrupted  by  random  disturbances  and  is  unlikely  to  be  predictable  in  detail  except  over  fairly  short 
time  periods.  As  a  result,  the  assessment  of  system  performance  and  the  exercise  of  control  seems  to  rest 
less  on  analytical  understanding  of  the  system  than  on  a  behavioral  one. 

Jowitt  argues  that  a  fuzzy  assessment  of  treatment  plant  dynamics  and  river  behavior  are  more 
important  means  of  control.  Fuzzy  logic  seeks  to  harness  the  treatment  plant  manager's  experience  and 
expertise  via  operational  control  actions  simulated  by  statements  such  as:  "If  discharge  level  is  'high', 
increase  sludge  waste  rate  by  a  'small'  amount".  Such  fuzzy  statements  are  preferred  to  the  more  analytical 
results  that  come  from  noise-corrupted  differential  equations.  The  author  argues  the  robustness  of  fuzzy 
logic  is  to  be  preferred  to  the  sophistication  of  differential  equation  models. 

Precise  but  inaccurate  models  can  be  a  problem  because  unknown  variations  (noise)  are  difficult  to 
embed  in  the  differential  equation  models.  "Fuzzification”  of  the  estimation  algorithm  is  required.  In  the 
author’s  opinion,  using  expert  system  and  fuzzy  logic  theory  to  develop  an  operating  philosophy  that  is  an 
extension  of  common  sense  is  to  be  preferred  over  uncertain  analytical  systems. 
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A20:  Wetland  Creation  and  Restoration:  The  Status  of  the  Science.  Edited  by  Jon  A.  Kusler  and 
Mary  E.  Kentula.  Washington,  D.C.:  Island  Press,  1990.  pp.  xxv,  595.  Twenty-eight  articles 
with  an  executive  summary.  ISBN  1-55963-044-2  paperback. 

Applicability 

This  book  does  not  explicitly  address  risk  and  uncertainty  issues  that  arise  in  the  creation  and 
restoration  of  wetlands.  It  addresses  so  many  of  these  issues  implicitly,  however,  that  it  proves  an 
invaluable  source  of  insight  into  the  nature  of  much  of  the  uncertainty  attending  wetland  creation  and 
restoration.  It  is  an  essential  element  of  any  library  on  wetlands  for  its  lists  of  literature  cited  after  each 
article  alone.  Fortunately,  most  of  the  articles  are  well-written  and  illustrated  and  full  of  valuable,  hard-to- 
find  information  as  well. 

This  collection  of  articles  by  authors  selected  for  their  expertise  in  particular  areas  of  wetlands 
science  or  their  active  involvement  in  actual  wetland  creation  or  restoration  meets  several  needs  at  once. 
It  provides  a  preliminary  evaluation  of  the  status  of  the  science  of  wetland  creation  and  restoration  in  the 
U.S.  As  such  it  is  a  valuable  introduction  to  all  parties  interested  in  this  topic.  It  provides  considerable 
detailed  technical  discussion  of  interest  to  the  scientific  community,  balanced  with  a  considerable  emphasis 
on  the  management  issues.  The  discussions  throughout  the  articles  constantly  and  consistently  highlight  the 
relevant  uncertainties  encountered  in  wetland  creation  and  restoration.  It  does  not  include  any  systematic 
discussion  of  habitat  models.  It  is  highly  recommended. 

Review 


Less  than  half  of  the  215  million  acres  of  wetlands  estimated  to  be  present  in  the  48  coterminous 
states  at  the  time  of  the  European  settlement  remain.  Each  year  an  estimated  450,000  acres  of  wetlands,  an 
area  12  times  the  size  of  the  District  of  Columbia,  are  destroyed.  Questions  about  how  well  restored  or 
created  wetlands  compensate  for  destruction  of  existing,  naturally  functioning  wetlands  assume  increasing 
significance  as  the  United  States  moves  toward  a  policy  of  no  net  loss  of  wetlands.  This  important  book 
addresses  the  state  of  scientific  knowledge  of  and  about  wetland  creation  and  restoration  in  two  volumes. 
It  will  help  planners,  analysts  and  managers  eliminate  the  uncertainty  about  what  it  is  we  are  uncertain  about. 
This  book  is  most  useful  for  its  ability  to  point  out  what  we  don't  know. 

The  first  volume  comprises  a  series  of  regional  reviews.  Each  of  the  16  regional  reviews 
summarizes  wetland  creation  and  restoration  experiences  in  broadly  defined  wetland  regions  (e.g.,  coastal 
plain  wetlands  in  Florida,  forested  wetland  vegetation  in  the  southeast).  The  reviews  summarize  the 
available  information  and  describe  what  has  and  has  not  been  learned.  These  discussions  are  invaluable  aids 
to  the  careful  reader  who  is  trying  to  learn  what  the  major  types  and  causes  of  uncertainty  about  wetland 
creation  and  restoration  are. 

The  second  volume  comprises  a  series  of  12  theme  papers  covering  a  wide  range  of  topics  of  general 
application  to  wetland  creation  and  restoration.  These  include  such  broad  themes  as  planning  (goals. 
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objectives,  and  so  on)  and  management  techniques,  as  well  as  specific  themes  such  as  wetland  zonation, 
vegetation  change,  and  moist-soil  impoundments  for  nonpersistent  emergents. 


Conclusions  from  these  articles  are  offered  in  a  profoundly  insightful  and  effective  executive  study. 
With  regard  to  the  adequacy  of  our  scientific  understanding  and  the  success  of  creation  and  restoration 
projects  in  meeting  project  goals,  following  are  some  conclusions  gleaned  from  the  articles. 

1)  Practical  experience  and  the  available  science  base  on  restoration  and  creation  are 
limited  and  vary  with  region  and  wetland  type. 

2)  Most  wetland  restoration  and  creation  projects  do  not  have  goals  specified.  This 
complicates  efforts  to  evaluate  the  success  of  the  project.  Partial  or  total 
revegetation  is  rarely  the  specific  goal  of  a  project  but  it  is  often  the  easiest  to 
measure. 

3)  Monitoring  of  wetland  and  creation  projects  has  been  uncommon.  There  has  been 
little  short-term  monitoring  and  less  long-term  monitoring. 

These  findings  have  significant  implications  for  the  Corps'  proposed  adaptive  management 
technique.  It  would  appear  that  such  techniques  will  be  blazing  trails  in  virgin  territory.  Those  blazing  such 
trails  will  be  buoyed  by  this  text’s  plethora  of  practical  advice.  This  includes  check  lists,  diagrams,  and 
suggestions  such  as  "It  is  not  enough  to  design  a  project  and  turn  it  over  to  traditional  construction 
personnel." 

With  regard  to  the  success  of  restoration  and  creation  projects,  the  articles  offer  the  following  points, 
among  many  others. 

1)  Restoration  or  creation  of  a  wetland  that  "totally  duplicates"  a  naturally-occurring 
wetland  is  impossible.  However,  some  systems  may  be  approximated  and 
individual  wetland  functions  may  be  restored  or  created.  This  does  not  mean, 
however,  that  it  will  have  habitat  or  other  values  equaling  those  of  a  natural 
wetland. 

2)  Partial  project  failures  are  common.  Total  failures  are  common  for  some  wetlands, 
like  sea  grass  and  certain  forested  wetland  areas.  Although  the  causes  differ, 
common  problems  are  identified  throughout  the  articles. 

3)  Success  varies  with  the  type  of  wetland  and  target  functions  including  the 
requirements  of  target  species.  There  appears  to  be  more  success  with  revegetation 
of  coastal,  estuarine,  and  freshwater  marshes  than  most  other  types.  There  are  very 
few  studies  concerning  the  use  of  restored  or  created  wetlands  by  particular  animal 
species. 

4)  The  ability  to  restore  or  create  particular  wetland  functions  varies  by  function.  That 
ability  depends  on  the  amount  of  available  scientific  knowledge  (fundamental 
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uncertainty  issues);  the  ease  and  cost  of  creating  or  restoring  certain  wetland 
characteristics  (topography  and  infiltration  capacity,  for  example);  and,  the 
probabilities  that  structural  characteristics  will  give  rise  to  specific  functions  (risk 
and  uncertainty  issues).  These  themes  can  be  found  throughout  the  articles. 

5)  Long-term  success  may  be  quite  different  from  short-term  success.  The  major 
threat  appears  to  be  hydrologic  fluctuations.  Other  threats  include  pollution, 
erosion,  off-road  traffic,  and  grazing. 

6)  Long-term  success  depends  upon  the  ability  to  assess,  recreate,  and  manipulate 
hydrology.  The  extent  to  which  hydrology  can  be  determined  and  established  is 
an  important  determinant  of  success.  The  authors  find  the  greatest  probability  for 
success  are,  in  order:  I)  estuarine  marshes;  ii)  coastal  marshes;  iii)  freshwater 
marshes;  iv)  isolated  marshes  supplied  by  surface  water;  v)  forested  wetland;  and, 
vi)  isolated  freshwater  wetlands. 

7)  Success  often  depends  upon  the  long-term  ability  to  manage,  protect,  and 
manipulate  wetlands  and  adjacent  buffer  areas.  These  are  considerable  insights  for 
the  Corps'  adaptive  management  approach  to  environmental  restoration  planning. 

Many  articles  make  the  point  that  original  design  specifications  may  prove 
insufficient  to  achieve  project  goals  and  midcourse  corrections  are  often  critical  to 
long-term  success. 

8)  Success  depends  upon  expertise  in  project  design  and  upon  careful  project 
supervision.  In  this  new  area  there  appears  to  be  no  substitute  for  experience. 

9)  "Cook  book"  approaches  for  wetland  restoration  or  creation  will  likely  be  only 
partially  successful.  The  basic  problem  appears  to  be  that  scientific  uncertainty  is 
too  great  to  enable  us  to  provide  rigid  guidance.  However,  guidance  suggesting 
ranges  of  conditions  conducive  to  success  are  both  desirable  and  possible. 

Many  of  the  authors  list  specific  research  needs  in  their  papers.  These  provide  valuable 
identification  of  major  and  significant  sources  of  uncertainty.  With  regard  to  filling  these  gaps  in  scientific 
knowledge  the  papers  lead  to  the  following  conclusions. 

1)  Systematic  monitoring  of  restoration  or  creation  projects  is  needed.  This  will 
require  careful  baseline  studies  and  monitoring  selected  features  of  the  new/restored 
systems  at  periodic  intervals. 

2)  Demonstration  projects  offer  the  greatest  promise  for  "control"  and  are  needed. 

3)  More  traditional  scientific  research  is  needed  on  a  wide  variety  of  topics. 

4)  Continued  synthesis  of  existing  scientific  knowledge  is  needed.  Too  much  of  what 
exists  is  fragmented  by  time,  geography,  and  money. 

The  collection  of  papers  offers  considerable  discussion  of  these  points  throughout  the  book.  In  the 
executive  summary  fourteen  recommendations  for  wetland  managers  are  offered.  These  recommendations, 
culled  from  the  papers,  are  clear  and  practical.  The  most  valuable  contribution  of  this  collection  of  papers 
is  the  major  step  they  take  in  identifying  the  major  sources  of  uncertainty  in  creating  and  restoring  wetlands, 
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managing  them,  and  assessing  their  success.  This  book  is  a  must  for  anyone  involved  in  wetlands  restoration 
and  creation. 
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All :  “Application  of  Event  Tree  Risk  Analysis  to  Fisheries  Management.”  By  Ernst  Linder,  G.P. 
Patil,  and  Douglas  S,  Vaughan.  In  Ecological  Modeling.  Vol.  36, 1987.  pp.  15-28. 

Applicability 

This  article  was  selected  because  of  its  fishery  application.  Fisheries  will  be  important  resources 
in  Corps  planning  and  this  article  provides  an  example  of  what  is  being  done  to  apply  risk  analysis  to 
fisheries.  The  authors  make  extensive  use  of  event  trees.  The  article  is  of  uneven  quality.  Though  the 
substance  is  sound,  the  article  has  many  spelling  and  grammatical  errors,  rather  unheard  of  in  the 
professional  literature.  We’re  not  sure  what  to  make  of  this  but  it’s  not  a  sign  of  strength. 

The  article  provides  a  good  example  of  how  continuous  distributions  can  be  discretized  to  keep  event 
trees  manageable.  The  article  does  not  follow  the  EPA  human  health  paradigm  for  quantifying 
environmental  risks  and  that  alone  makes  this  article  worth  a  look  for  anyone  dealing  with  risk  analysis  of 
fisheries. 

Review 


In  this  article  risk  analysis  is  defined  as  the  evaluation  of  the  probability  of  end  events,  such  as 
maximum  sustainable  yield,  in  terms  of  sequences  of  earlier  events,  such  as  stock-recruitment.  In  fisheries 
science  and  management,  the  biological/ecological  variability  and  uncertainty  are  very  large  compared  to 
the  uncertainties  encountered  in  the  physical  and  engineering  sciences.  Estimating  fishery  populations  and 
harvest  levels  are  difficult  tasks.  In  this  article  the  authors  use  decision/event  trees  to  structure  the  risk 
analysis.  Two  examples  are  offered:  the  Georges  Bank  haddock  fishery;  and  the  Gulf  of  Mexico  menhaden 
fishery. 

In  Georges  Bank  increased  fishing  effort  and  reduced  stock  size  led  to  a  dramatic  decrease  in  the 
fishery.  By  1980  scientists  concluded  the  catch  must  be  halved  to  restore  the  fisheiy  to  pre-1960  levels.  The 
catch  was  not  restricted  so  the  survival  of  the  fishery  becomes  an  important  question.  The  authors  propose 
the  use  of  risk  analysis  to  evaluate  alternative  management  strategies.  Virtual  population  analysis  (VPA) 
was  used  to  estimate  the  size  of  year  classes  after  the  fact.  Stock-recruitment  models  based  on  VPA  values 
are  used  to  construct  a  probability  event  tree  that  assesses  the  probability  of  recovery  of  the  fishery. 
Recovery  is  defined  as  two  strong-year  classes  during  the  three-year  period  from  two  to  four  years  after  the 
current  year. 

Survival  depends  on  mortality  as  well.  Mortality  and  annual  spawning  stock  depend  on  fishing 
effort.  The  authors  evaluated  two  choices  for  fishing  mortality:  the  fishing  effort  that  results  in 

maximum  yield  per  recruit;  and,  2)  Fo  i,  the  fishing  effort  that  maintains  stock  sizes  above  the  lowest  quintile 
of  previously  observed  stock  sizes. 

Traditional  event  tree  analysis  presents  a  specific  point  estimate  with  a  separate  probability  for  each 
event.  The  authors  divided  the  probability  density  function  into  three  interval  estimates:  upper  (^lo), 
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middle  (-la  to  +lo),  and  lower  (^-lo).  In  fact,  the  authors  did  not  go  beyond  ±2a,  covering  about  95%  of 
the  potential  observations.  The  distribution  was  normalized  to  probabilities  of  0.1,  0.8,  and  0.1.  The 
intervals  keep  the  event  tree  to  a  manageable  size.  The  probability  of  two  high  recruitment  years  out  of  three 
when  fishing  effort  is  was  0.549.  For  Fq.,  the  probability  was  0.876.  A  similar  analysis  is  offered  for 
the  Gulf  menhaden. 
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A22:  "Risk  Analysis  for  Conservation  Biologists."  By  Lynn  A.  Maguire.  Conservation  Biology.  Vol. 
5,  No.  1,  March  1991.  pp.  123-125. 

Applicability 

This  brief  article  provides  a  handy  introduction  to  a  dilemma  faced  by  many  conservation  biologists. 
Trained  as  scientists  they  are  loathe  to  take  a  strong  stand  on  an  issue  until  sufficient  data  are  available  to 
support  that  stand.  Unfortunately,  waiting  can  limit  the  range  of  options.  Risk  analysis  is  suggested  as  a 
solution  to  this  dilemma. 

In  environmental  resource  planning  Corps  analysts  are  frequently  dealing  with  agencies  and  other 
stakeholders  who  act  before/without  sufficient  data  for  scientific  certainty.  Their  actions  are  often  founded 
in  legislative  or  regulatory  standards.  One  might  infer  from  this  article  that  risk  analysis  could  improve  this 
coordination  process  portion  of  the  decision  process. 

This  is  an  easy  and  quick  read  with  several  interesting  ideas  for  those  new  to  the  issue  of  risk 
analysis  in  the  environmental  resource  arena.  Of  all  the  areas  of  risk  analysis  reviewed  as  part  of  the 
preparation  of  this  report,  risk  analysis  for  conservation  biologists  appears  to  be  the  subfield  most  closely 
aligned  with  the  needs  of  environmental  resource  analysts.  This  article  is  recommended  for  those  seeking 
a  general  orientation  to  some  of  the  issues  and  general  approaches  to  risk  analysis. 

Review 


The  article  begins  with  the  definition,  "Conservation  biology  has  been  described  as  a  crisis 
discipline,  where  limited  information  is  applied  in  an  uncertain  environment  to  make  urgent  decisions  with 
sometimes  irrevocable  consequences"  (p.l23).  Substitute  environmental  mitigation,  ecosystem  restoration, 
or  environmental  resource  planning  for  conservation  biology,  and  you  have  a  functional  definition  with 
which  many  analysts  would  readily  agree. 

The  crux  of  the  article  is  that  scientific  training  frequently  prevents  biologists  from  speaking  out  on 
issues  before  all  the  evidence  is  available.  Reluctance  to  initiate  captive  breeding  for  the  California  condor 
and  the  black-footed  ferret  reached  the  point  where  continued  inaction  would  have  resulted  in  a  disaster.  By 
delaying  implementation  to  the  last  moment  the  prospects  for  success  of  these  programs  were  nearly  lost. 

How  can  conservation  biologists  balance  their  commitment  to  scientific  excellence  and  their 
responsibilities  for  solving  environmental  problems?  Risk  analysis,  the  author  suggests,  may  be  the  answer. 
Considering  the  uncertainties  inherent  in  a  decision  problem  in  a  systematic  way  clarifies  choices.  What's 
the  range  of  things  that  could  be  said  about  a  management  decision?  What  mistakes  could  be  made  in  what 
is  said?  What  are  the  consequences  of  being  right  or  wrong  in  terms  of  management  actions  taken  or  not 
taken,  changes  in  the  status  of  environmental  and  social  consequences,  professional  status,  and  impacts  on 
funding  or  employment?  The  author  asks  how  do  we  regulate  air  pollution  to  protect  forests  when  we  don’t 
know  the  effects  of  air  pollution  on  forests?  An  interesting  question  with  many  parallels  indeed. 
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The  author  presents  a  simple  decision  tree  that  artfully  demonstrates  how  the  uncertainty  in  structure 
of  the  decision  problem  can  be  illustrated.  She  considers  consequences  to  the  forests,  budgets,  and  the 
scientist's  status.  The  advantages  of  this  approach  are:  1)  it  is  personalized;  2)  it  brings  our  actions  more  in 
line  with  our  beliefs;  3)  it  helps  us  avoid  unnecessarily  conservative  positions;  and  4)  it  helps  us  distinguish 
between  the  sphere  of  the  analyst  and  the  decision  maker. 
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A23:  "Risk  Analysis  of  Multiple  Environmental  Factors;  Radiation,  Zinc,  Cadmium,  and 
Calcium."  By  Junko  Matsubara,  Kuiaki  Ishioka,  Yoshisada  Shibata,  and  Kazuaki  Katoh. 
Environmental  Research.  No.  40, 1986.  pp.  525-530. 

Applicability 

One  of  the  most  vexing  problems  of  any  risk  analysis  is  how  to  account  for  the  complex  interactions 
and  interdependencies  of  multiple  factors.  Most  risk  analyses  still  treat  factors  as  largely  independent  of  one 
another.  This  article,  though  developed  for  the  health  risk  analysis  field,  does  offer  a  cogent  example  of  how 
one  might  approach  the  problem  of  interdependencies  among  factors.  Though  the  authors  allude  to  the 
combination  of  variables  in  interaction  terms  they,  unfortunately,  side-step  the  opportunity  to  do  so  in  this 
article.  This  article  is  recommended  to  readers  interested  in  how  one  might  combine  the  effects  of  several 
variables  in  an  analysis.  The  technique  used,  a  multiple  logistic  model,  typically  requires  a  graduate  level 
exposure  to  econometric  or  statistical  analysis. 

Review 


Environmental  problems  are  the  net  result  of  many  influences.  Adult  diseases  are  associated  with 
multiple  risk  factors  that  have  been  difficult  to  evaluate.  The  authors  suggest,  "..it  is  better  to  take  several 
risk  factors  into  account  simultaneously  for  considering  environmental  problems,  though  the  traditional 
scientific  approach  is  to  focus  on  a  single  risk  factor." 

This  paper  reports  the  results  of  a  laboratory  experiment  in  which  360  mice  were  exposed  to  four 
risk  factors  at  once.  The  risk  factors  were  gamma  radiation  in  intestinal  doses;  zinc  through  drinking  water; 
cadmium  by  spot  injection;  and  a  calcium-deprived  diet.  The  mortality  rates  were  noted  and  the  experiment 
was  repeated  with  another  group  of  360  mice. 

The  mortality  rate  of  a  mouse  within  30  days  was  modeled  using  a  multiple  logistic  model  as 

follows: 

6  =  {1  +  exp(-  Po  -  Pi^i  -  P2X2  -  P3X3  -  P4X4)  ‘} 


0  takes  the  value  0  if  the  mouse  lives  and  1  if  it  dies.  The  x  variables  indicate  the  extent  of  exposure  to  the 
four  factors  considered.  The  results  showed  crude  mortality  rates  of  5.83%  for  the  first  group  of  mice  and 
5.23%  for  the  second  group.  Radiation  was  found  to  contribute  most  to  lethality  and  zinc  tended  to  reduce 
the  lethality  of  the  gamma  radiation.  Interaction  terms  in  the  form  XjXj  could  be  added  to  the  model  but  this 
was  not  done.  When  such  terms  are  added,  interpretation  of  partial  derivatives  can  be  difficult.  However, 
this  was  an  important  if  unexploited  advantage  of  this  model. 

The  authors  conclude  the  article  with  the  following  point:  "Therefore  we  assume  that  multiple  risk 
analysis  using  a  multiple  logistic  model  works  for  effective  sum-up  of  the  data  from  a  multivariate 
experimental  system  and  provides  tentative  estimates  for  risk  prediction"  (p.  530). 
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A24:  "The  Marginal  Cost  of  Species  Preservation:  The  Northern  Spotted  Owl".  By  Claire  A. 

Montgomery,  Gardner  M.  Brown,  Jr.,  and  Darius  M.  Adams.  Journal  of  Environmental 

Economics  and  Management.  Vol.  26,  pp.  111-128, 1994. 

Applicability 

This  article  is  the  conceptual  offspring  of  several  ideas  presented  in  other  works  reviewed  in  this 
annotated  bibliography.  This  article  provides  an  excellent  example  of  how  interdisciplinary  analysis  brings 
together  many  of  the  concepts  separately  developed  in  the  literature.  Bishop's  (1978)  economics  article 
(reference  A6)  and  Burgman's  (1993)  biology  book  (reference  A7),  reviewed  elsewhere  in  this  report, 
provide  diverse  perspectives  on  complex  issues.  Montgomery  et  al.  integrate  the  perspectives  of  both 
disciplines  in  an  effective  manner  that  provides  an  excellent  model  for  Corps  planners.  This  is  not  a 
multidisciplinary  approach  to  the  problem.  It  is  an  interdisciplinary  approach  that  truly  integrates  the 
perspectives  of  the  two  disciplines. 

This  paper  presents  a  marginal  cost  of  survival  curve  that  could  provide  an  extremely  valuable  model 
for  the  Corps  to  consider  for  its  decision  process.  The  article  will  be  difficult  for  non-economists  and  for 
those  who  do  not  have  some  calculus  background.  It  is  nonetheless  well  worth  the  effort  for  these  people 
as  well  as  economists  and  others  with  some  mathematical  background.  It  provides  an  example  of  a  model 
that  may  well  be  one  of  the  more  promising  ideas  on  the  horizon  for  environmental  resource  planners. 

Review 


The  authors  begin  by  posing  the  question,  “Which  species  should  we  attempt  to  save  and  what  means 
should  we  employ?”  To  answer  the  first  question  we  must  understand  the  genetic  and  economic  uniqueness 
of  the  species  and  its  contribution  to  biodiversity.  How  might  this  species  contribute  to  the  survival  of  other 
species?  What  is  its  ecosystem  function?  Answering  the  second  question  requires  us  to  estimate  the  relative 
merits  and  costs  of  alternative  preservation  methods. 

In  the  U.S.,  the  evaluations  are  currently  made  implicitly,  rather  than  explicitly,  by  the  U.S. 
Department  of  Interior  Fish  and  Wildlife  Service  (FWS)  within  the  legal  constraints  of  the  Endangered 
Species  Act.  The  authors  argue  that  the  analysis  of  economic  trade-offs  inherent  in  these  issues  is  the 
intellectual  turf  of  economists.  The  question  most  often  asked  of  economists  in  planning  contexts  where  a 
species  has  been  identified  as  a  valuable  component  of  diversity  (or,  perhaps,  as  an  indicator  species)  is, 
"What  is  the  cost  to  ensure  its  survival?"  This,  the  authors  say,  is  the  wrong  question.  The  right  question  is 
not  how  much  will  it  cost  to  ensure  survival  but,  what  is  the  probability  that  the  species  will  survive. 

So  framing  the  question,  the  authors  argue  that  certainty  of  survival  is  not  a  realistic  approach  to  the 
issues  raised  above.  The  probability  of  survival  is  the  appropriate  way  to  approach  the  measurement  of 
project  outputs  with  regard  to  targeted  species. 
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Models  of  species  population  dynamics  and  relevant  markets  were  linked  to  derive  a  marginal  cost 
curve  for  species  survival  using  the  northern  spotted  owl.  The  current  controversy  attending  the  northern 
spotted  owl  is  summarized  effectively  before  the  net  economic  trade-offs  surrounding  the  owl  issue  are 
quantified  through  the  development  of  a  supply  curve. 

The  basic  model  used  is; 

S(C)  =  Prob  [Y(To)^Y„JC] 

where  C  is  the  capacity  of  protected  owl  habitat,  measured  in  owl  pairs,  that  protected  habitat  is  capable  of 
supporting  if  fully  occupied;  Y(T)  is  population  at  time  T;  To  is  a  chosen  time  horizon;  and  Y„i„  is  a  chosen 
threshold  population  size.  Most  spotted  owl  biologists  agree  that  timber  harvest  and  protection  of  habitat 
for  nesting  owl  pairs  are  mutually  exclusive  or  at  least  highly  conflicting.  Hence,  objectives  of  social  benefit 
estimation  require  that  specific  tracts  of  land  be  allocated  to  one  use  or  the  other. 

Potential  welfare  losses  in  wood  products  markets  (softwood  lumber,  plywood,  and  stumpage)  were 
estimated  using  the  Timber  Assessment  Market  Model.  These  losses  in  billions  of  1990  dollars  were  , 
estimated  for  various  reductions  in  annual  public  timber  harvests.  In  addition,  the  probability  of  the  owl 
population  exceeding  the  minimum  safe  standard  (see  Bishop  1978,  reference  A6)  at  the  end  of  150  years 
as  a  function  of  habitat  capacity  was  estimated.  The  annual  reduction  in  public  timber  harvest  in  board  feet 
for  each  additional  pair  of  protected  owl  pairs  was  also  estimated  as  a  function  (i.e.,  the  marginal  physical 
cost  curve). 


Using  the  marginal  physical  cost  curve  as  the  common  linkage,  a  marginal  cost  curve  for  owl 
survival  is  estimated.  This  curve  shows  the  cost  in  billions  of  1990  dollars  of  increasing  the  probability  of 
owl  survival  by  one  percentage  point.  Results  are  presented  for  selected  conservation  strategies.  The 
Interagency  Scientific  Committee  (ISC)  proposal  would  result  in  a  $33  billion  welfare  loss  in  return  for  a 
91%  chance  the  owls  would  still  survive  after  150  years.  Increasing  the  chance  of  survival  from  90%  to 
91%  means  an  additional  cost  of  $1.4  billion. 

The  technique  is  promising  because  it  supports  the  type  of  marginal  cost  analysis  currently  favored 
for  environmental  resource  planning.  It  allows  a  different  kind  of  trade-off  from  the  current  cost  per  habitat 
unit.  Unfortunately,  the  models  used  in  this  paper  do  not  obviate  the  need  for  controversial  studies.  The 
most  controversial  portion  of  the  subject  study  "..appears  to  be  the  biological  relationship  between  habitat 
capacity  and  the  probability  of  owl  survival."  Thus,  the  habitat  models  will  continue  to  occupy  a  position 
on  the  critical  path  toward  improving  the  risk  and  uncertainty  analysis  for  environmental  resources  planning. 
The  authors  astutely  point  out  that  it  is  likely  to  be  politically  difficult  to  enact  a  policy  that  imposes  a  heavy 
toll  on  one  region  for  the  general  benefit  of  all. 
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A25:  "Fuzzy  Multigroup  Conflict  Resolution  for  Environmental  Management."  By  G.  Munda,  P. 

Nijkamp,  and  P.  Rietveld.  In  The  Economics  of  Project  Appraisal  and  the  Environment  edited 
by  John  Weiss.  Aldershot,  England:  Edward  Elgar  Publishing  Limited,  1994.  pp.  161-183. 
ISBN  1  85278  678  7. 

Applicability 

This  article  was  selected  in  order  to  demonstrate  how  extensions  of  fuzzy  set  theory  are  being  used 
to  address  issues  of  uncertainty.  It  suffers  from  a  lack  of  detail  that  the  authors  justify  by  referring  the  reader 
to  their  other  articles.  This  will  frustrate  the  reader  unfamiliar  with  the  literature  on  fuzzy  set  theory  and 
coalition  formation.  Nonetheless,  the  article  does  provide  the  reader  entree  into  these  two  new  fields  of 
theory.  Thankfully,  a  case  study  is  provided  that  does  suggest  how  these  techniques  can  be  used  to  evaluate 
environmental  projects  using  multi-criteria  techniques.  The  reader  won't  be  able  to  read  this  article  and  turn 
around  and  apply  the  techniques,  but  he  will  know  there  is  a  technique  out  there  that  could  well  suit  his 
purposes  in  many  situations.  This  could  be  enough  to  send  the  reader  on  a  search  for  the  additional 
referenced  articles  or  someone  experienced  in  these  techniques.  The  article  is  recommended  for  making  the 
reader  familiar  with  the  existence  of  these  techniques. 

Review 


Evaluation  models  are  used  to  judge  the  feasibility  and  desirability  of  alternative  courses  of  action. 
They  can  be  monetary  or  non-monetary  but  the  necessity  of  analyzing  conflicts  among  policy  objectives  in 
the  case  of  environmental  issues  have  led  to  a  need  for  more  appropriate  tools.  The  authors  suggest  multiple- 
criteria  evaluation  techniques  provide  such  tools.  Qualitative  multi-criteria  methods  for  environmental 
management  using  a  mix  of  quantitative  and  qualitative  information,  also  called  mixed  information 
problems,  are  discussed,  but  the  emphasis  is  on  "..a  discrete  multi-criteria  method  whose  impact  (or 
evaluation)  matrix  may  include  either  crisp,  stochastic  or  fuzzy  evaluations  of  the  performance  of  an 
alternative  with  respect  to  a  criterion  g^,.” 

The  main  steps  of  this  method  include;  1)  definition  of  a  fuzzy  region  of  satisfactory  alternatives; 
2)  comparison  of  fuzzy  sets;  3)  pairwise  comparison  of  alternatives;  and  4)  evaluation  of  alternatives. 
Unfortunately,  the  article  does  not  include  nearly  enough  detail  to  appreciate  the  steps.  Additional  details 
are  provided  in  other  articles  written  by  Munda.  Fortunately,  however,  a  very  nicely  developed  example  is 
presented  later  in  the  article.  The  reader  may  not  appreciate  the  mathematical  rigor  of  the  approach  but  it's 
applicability  is  very  evident. 

After  introducing  the  steps  of  their  method  the  authors  provide  a  concise  overview  of  coalition 
formation  theory.  The  aim  of  this  theory  is  to  predict  a  set  of  coalitions  that  are  likely  to  be  formed  in  favor 
of  or  opposition  to  alternatives  in  a  given  political  situation.  The  overview,  though  sparse,  provides  an 
introduction  to  several  applied  game-theoretic  principles  including  size  theory,  policy  theory,  and  its  two 
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variants,  minimal  range  theory  and  policy  distance  theory.  The  discussion  of  the  model  and  coalition 
formation  theoiy  are  likely  to  frustrate  readers  not  familiar  with  fuzzy  set  theory  and  game  theoretics.  If  they 
prevail  to  the  illustration  of  the  method,  their  effort  will  be  rewarded. 

The  case  study  considers  a  marl  winning  process  in  the  Netherlands.  Marl  winning  is  part  of  the 
concrete  production  process.  Marl  winning  destroys  many  of  the  original  ecosystem  values.  Thirteen 
evaluation  criteria  are  developed  for  economic,  social  and  environmental  values.  Seven  alternative  actions 
are  developed  and  an  evaluation  matrix  for  a  fuzzy  land  use  problem  is  presented.  The  alternatives  are 
ranked  for  criteria  like  national  production  of  marl,  tourist  attractiveness,  and  diversity  and  scarcity  of  eco- 
and  bio-components  using  a  fuzzy  linguistic  scale.  In  other  words,  because  there  is  a  dearth  of  quantitative 
data,  each  alternative  is  ranked  for  each  criteria  as  either  bad,  moderate,  good  or  excellent. 

In  addition  to  this  evaluation  matrix  six  interest  groups  are  identified.  Fuzzy  evaluations  of 
alternatives  according  to  each  interest  group  are  provided  using  terms  like  bad,  fairly  bad,  moderate,  fairly 
good,  good,  and  very  good.  The  fuzzy  multi-criteria  procedure  proposed  for  each  pairwise  comparison 
proceeds  as  follows.  Three  statements  for  each  pairwise  comparison  are  considered:  a,  is  better  than  a2, 
a,  and  are  indifferent;  and  a,  is  worse  than  %.  A  degree  of  truth  is  expressed  for  each  statement  by  the 
evaluator.  That  is,  they  indicate  that  the  statement  is  either  true,  t  =  1,  or  the  statement  is  false,  t  =  0. 

On  the  basis  of  these  pairwise  comparisons  (it  is  not  explained  how  group  judgments  are  arrived  at) 
between  alternatives,  the  alternatives  are  ranked  in  sets.  For  example,  in  this  example  (a^,  a,,  a^,  a7)>(a|,  az, 
aj),  where  the  a,  are  the  alternatives.  Using  a  semantic  distance  function  a  similarity  matrix  is  developed  that 
shows  how  similar  the  various  interest  groups  are  to  one  another  in  their  views.  The  degree  of  similarity  is 
subsequently  used  to  predict  coalitions  that  can  be  expected  to  form  at  various  similarity  thresholds. 

The  article  provides  an  interesting  example  that  will  suggest  many  analogous  situations  for  Corps 
planners.  The  fuzzy  set  approach  illustrated  here  provides  the  reader  with  an  introduction  to  the  types  of 
things  that  can  be  done  to  address  the  uncertainty  that  will  inevitably  arise  in  environmental  projects. 
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A26:  “The  Values  of  a  Habitat.”  By  Kelly  Parker.  Environmental  Ethics.  Vol.  12.  No.  4.  Winter 
1990.  pp.  353-368. 

Applicability 

This  paper  has  a  “don’t  miss”  title.  Environmental  resource  planning  is  currently  veiy  much  habitat- 
oriented.  The  value  of  habitat  is  one  of  the  fundamental  uncertainties  of  the  planning  process. 
Unfortunately,  for  the  analyst  seeking  pragmatic  advice,  this  philosophical  piece  provides  none  of  that.  The 
author  develops  a  notion  of  pragmatic  environmental  thinking  in  a  paper  that  has  nothing  to  offer  the  risk 
analyst,  but  a  great  deal  to  offer  those  who  want  to  contemplate  the  true  value  of  habitat.  Generally,  we  find 
it  a  good  think  piece  that  has  little  pragmatic  value  to  planners  in  this  form.  The  author  says  if  we  can  get 
people  on  different  sides  of  an  issue  to  see  that  we  all  must  experience  both  the  adequacy  and  significance 
of  a  habitat,  then  we  have  the  common  ground  we  need  to  work  out  some  solutions.  We’re  not  convinced 
this  framework  has  the  appeal  to  overcome  the  emotional  arguments  of  a  spotted  owl  conflict,  but  you  may 
think  differently. 

Review 


Technology  has  changed  the  balance  between  humans  and  their  environment.  Parker  argues  that  we 
need  a  framework  for  determining  the  principal  value  of  an  environment.  An  important  first  step  in 
developing  that  framework  is  to  determine  what  it  is  that  makes  a  place  important.  The  author  claims  an 
anthropometric  perspective  to  environmental  thinking  and  begins  the  paper  by  examining  two  other 
anthropometric  theories:  environmental  preservation  and  judicious  development.  A  major  distinction 
between  the  two  theories  is  who  has  the  burden  of  proof  The  preservationist  places  this  burden  on  those 
who  would  infringe  on  natural  systems.  The  developer  places  the  burden  on  those  who  would  protect  a 
resource  from  use.  Proof  is  a  fundamental  exercise  in  reducing  uncertainty.  Hence,  in  spirit  this  article  does 
relate  to  the  issues  of  risk  and  uncertainty. 

Parker  reviews  three  arguments  used  by  preservationists  against  developers.  After  offering  his  own 
preservationist  views  he  concludes  that  “..we  must  find  some  common  ground  on  which  we  can  discuss  the 
reasons  each  of  us  has  for  putting  forth  such  considerations.”  The  considerations  he  refers  to  are  our  views 
for  preservation  or  development. 

The  author  believes  pragmatic  environmental  thinking  provides  that  common  ground.  “The  point 
of  pragmatic  environmental  thinking  is  to  work  within  the  human  experience  of  the  world,  but  to  expand  the 
scope  and  meaning  of  that  experience  beyond  the  usual  level  of  dull  inattentiveness  which  fosters 
environmental  exploitation  and  destruction.”  The  author  contends  there  is  a  real  connection  between  the 
human  world  and  the  environment  and  argues  that  we  are  not  really  distinguishable  from  whatever 
environment  we  find  ourselves  in. 
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Section  1 002  is  a  1 .5  million-acre  stretch  of  tundra  that  will  either  become  part  of  the  Arctic  National 
Wildlife  Refuge  or  it  will  be  opened  to  oil  companies  for  exploratory  drilling;  a  classic 
preservationist/developer  clash.  Parker  argues  that  few  of  us  have  any  experience  with  a  wilderness  area  like 
Section  1002  as  a  habitat.  To  get  people  to  value  an  area  like  this  as  a  habitat  we  must  discover  some  basis 
in  other  aspects  of  our  common  experience.  He  then  draws  an  analogy  between  urban  and  natural  ecologies. 

An  urban  habit  must  be  adequate  for  life,  but  it  also  must  be  a  significant  habitat  for  life.  Adequacy 
refers  to  meeting  our  basic  needs  for  health,  safety  and  nutrition.  Significance  means  it  must  nourish  our 
spirits  as  well.  The  Earth  must  also  be  biologically  adequate  and  spiritually  significant.  The  common 
ground  for  the  developer  and  preservationist  is  our  common  human  way  of  experiencing  an  environment. 
“The  key  to  finding  common  ground  is  to  sensitize  the  motorboaters  to  the  affective  and  biological  role  of 
woods  in  their  experience  of  the  world,  and  likewise  to  sensitize  the  environmentalists  to  the  roles  of  the 
pleasure  boat  industry  in  their  experience  of  the  world.”  Parker  presents  his  framework  as  one  that  can 
overcome  impasses  like  Section  1002  and  others.  Considerations  of  adequacy  and  significance  can,  he 
maintains,  help  us  resolve  conflicting  positions  on  a  common  ground.  Any  idea  that  offers  help  in  resolving 
complex  conflicts  is  worthy  of  additional  consideration. 
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A27:  “Characterization  and  Evaluation  of  Uncertainty  in  Probabilistic  Risk  Analysis”.  By  G.W. 
Parry  and  P.W.  Winter.  Nuclear  Safety.  Vol.  22,  No.  1,  January-February  1981.  pp.  28-42. 

Applicability 

This  article  was  selected  on  the  strength  of  its  title,  which  promises  some  insight  in  the  handling  of 
uncertainty  in  risk  analysis.  It  delivers  that,  but  not  in  the  most  useful  form  for  the  typical  reader  of  this 
review.  One  must  really  know  the  risk  analysis  and  probability  literature  to  appreciate  this  article.  It  is 
important,  however,  in  that  it  presents  some  of  the  thinking  about  uncertainty  issues  that  was  taking  place 
in  the  nuclear  safety  industry  over  a  decade  ago.  Reading  this  article’s  description  of  probability  requires 
time  and  work.  It’s  a  difficult  discussion  for  most  analysts,  but  it  is  as  easy  a  discussion  of  some  of  the  basic 
issues  in  subjective  probability  that  will  be  found  in  the  scientific  literature.  This  article  is  not  recommended 
to  the  novice  in  risk  analysis. 

Review 


Probabilistic  risk  analysis  for  three  major  studies  in  the  nuclear  field  serve  as  the  starting  point  for 
this  article.  Results  of  these  risk  analyses  were  plotted  as  a  frequency  of  occurrence  of  an  accident  sequence 
versus  the  accident’s  particular  consequence.  These  figures  display  a  level  curve  along  which  the  expected 
value  of  the  risk  is  constant.  During  the  three  studies,  however,  significant  uncertainties  were  encountered 
in  both  the  frequencies  and  the  consequences  of  events. 

One  of  the  major  problems  encountered  in  the  risk  assessment  of  nuclear  plants  is  that  many  of  the 
event  sequences  of  interest  are,  by  design,  rare.  It  is  impossible  to  build  a  data  base  that  could  be  used  to 
make  inferences  within  a  reasonable  time  frame.  The  principle  technique  applied  in  these  instances  was  the 
combined  event  and  fault-tree  decomposition  of  accident  sequences  into  more  basic  events.  Frequently,  data 
were  available  for  these  more  frequent  events  so  the  frequency  of  the  rare  events  could  be  built  from  them. 

The  sources  of  uncertainty  found  in  these  studies  has  been  categorized  as  model  and  data  uncertainty. 
The  discussion  that  ensues  on  these  points  is  not  nearly  as  informative  as  that  offered  by  Morgan  1992, 
described  elsewhere  in  this  report  (reference  G13).  The  authors  do  indicate  some  important  limitations  of 
fault-trees,  however.  First,  it  is  often  not  possible  to  be  sure  that  all  events  leading  to  the  “top”  of  the  tree 
have  been  identified.  Thus,  completeness  is  a  major  problem.  Some  of  the  other  problems  identified  in  the 
article,  such  as  multistate  logic  and  external  events,  have  long  since  been  handled  by  the  fault-tree  tools. 
Thus,  the  article  suffers  from  its  age.  One  of  the  major  problems  identified  is  the  fact  that  the  probabilities 
of  events  are  uncertain. 

The  article  hits  its  stride  in  the  discussion  of  the  subjective  theory  of  probability.  Though  difficult 
for  those  unfamiliar  with  the  probability  literature,  it  is  a  valuable  contribution  to  the  risk  analysis  literature 
if  for  no  other  reason  than  the  many  classical  references  offered  by  the  authors.  Anyone  who  wants  an 
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introduction  to  the  classical  and  seminal  works  in  probability  theory  could  do  much  worse  than  to  start  with 
this  article. 

The  authors  favor  a  Bayesian  approach  to  probabilistic  risk  analysis.  The  experience  of  the  nuclear 
industry  with  a  lack  of  data  has  lead  the  authors  to  believe  that  the  correct  framework  for  assessing  coherent 
probabilities  is  Bayesian.  Degree  of  belief,  coherence  and  exchangeability  of  events  are  identified  as  the 
most  important  features  of  subjectivist  probability  theory.  Each  is  discussed  briefly  and  is  carefully 
footnoted. 

After  offering  the  briefest  of  overviews  on  Bayes  theory  (Baird  1989,  reviewed  elsewhere  in  this 
report  (reference  Gl),  does  a  much  nicer  Job  in  his  book),  the  authors  address  the  two  Bayesian  schools  of 
thought  on  the  choice  of  a  prior:  subjective  and  objective.  Given  the  evidence  that  exists,  should  two 
different  people  have  the  same  prior  (objective),  or  is  it  permissible  for  them  to  have  different  priors 
(subjective)?  There  follows  a  nicely  developed,  but  short,  discussion  of  the  principle  of  stable  estimation, 
the  principle  of  indifference,  invariance  theory,  principle  of  maximum  entropy,  the  method  of  Box  and  Tiao, 
and  the  method  of  Novick.  These  represent  issues  confronted  and  approaches  to  resolving  these  issues  in 
formulating  “ignorance”  priors.  These  topics  will  generally  be  a  bit  esoteric  for  the  needs  of  most  Corps 
analysts. 

The  discussion  of  the  quantification  of  uncertainty  that  follows  is  difficult.  Most  readers  would  be 
better  served  by  consulting  the  Baird  1989  text  (reference  Gl).  Advanced  probability  students  will  be  able 
to  appreciate  the  discussion  of  Bayesian  confidence  intervals  and  tolerance.  The  authors  found  a  major 
shortcoming  in  the  fact  that  there  are  apparently  no  methods  capable  of  handling  both  tolerance  and 
confidence.  Therefore,  physical  and  statistical  uncertainty  are  often  combined  in  single  makeshift  measures. 

The  authors  conclude  that  a  Bayesian  framework  is  best  for  rare  events  when  data  are  lacking  and 
expert  opinion  is  a  major  source  of  information.  These  sound  like  the  circumstances  Corps  planners  will 
often  face. 
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A28:  “First  Things  First:  Needed  -  A  Genetics  Risk  Analysis  Science.”  By  Jeremy  Rifkin.  The 
Environmental  Forum.  Volume  3,  Number  10.  February,  1985.  pp.  39-41. 

Applicability 

What  would  be  the  harm  in  using  some  new  plant  genetically  engineered  to  mature  quickly  to 
provide  cover  for  some  little  critter?  We  don’t  know.  That  is  precisely  the  author’s  point  in  this  article.  The 
argument  raised  may  be  a  bit  esoteric  at  first  glance,  but  it  is  likely  that  at  some  point  a  Corps  District  may 
be  faced  with  the  need  to  address  this  issue.  When  it  comes  up,  Rifkin  has  raised  a  few  points  that  may  help 
analysts  begin  to  think  about  this  fascinating  issue. 

Review 


“Biologists  are  able  to  recombine  genetic  material  from  unrelated  species,  and  in  so  doing  create 
modified  and  novel  forms  of  life  that  have  never  previously  existed  in  nature.”  This  is  a  direct  quote  from 
Rifkin’s  article,  one  of  many  that  gives  the  reader  reason  to  pause  and  reflect.  Rifkin  says  biotechnology 
is  about  to  replace  the  pyrotechnology,  that  has  had  us  heating,  melting,  forging,  shaping,  welding,  and 
cooking  inert  materials  into  new  shapes  and  forms  to  transform  life  on  earth.  Dismayed  that  government 
seems  to  hone  in  on  questions  of  how  to  proceed  and  how  to  regulate  bio-engineered  products  rather  than 
whether  to  do  it,  the  author  suggests  that  we  need  a  risk  assessment  methodology  to  judge  the  potential  risk 
of  these  products.  In  the  absence  of  such  a  methodology  he  believes  we  should  reject  the  introduction  of  any 
genetically  engineered  products  into  the  environment. 

Though  this  article  is  woefully  short  for  addressing  such  weighty  issues,  Rifkin  does  a  good  job  of 
raising  some  disturbing  questions.  First,  he  points  out  that  genetically  engineered  products  are  different  from 
the  petrochemical  products  he  uses  for  comparison  in  important  ways.  Because  they  are  alive  they  are  much 
more  unpredictable.  It’s  far  more  difficult  to  assess  all  the  potential  impacts  a  product  might  have  on  the 
environment.  These  products  have  the  ability  to  reproduce,  grow  and  migrate.  Once  released  they  are 
virtually  impossible  to  recall.  As  a  result,  the  potential  long-term  risks  to  the  environment  are  unknown  and 
great.  Rifkin  suggests  that  if  only  a  small  number  of  these  products  are  harmful  the  sheer  volume  of  new 
bioengineered  products  virtually  ensures  that  some  will  become  problems. 

A  second  set  of  problems  is  that  genetic  technology  as  applied  to  agriculture  and  animal  husbandry 
is  designed  to  increase  the  speed  of  maturation  and  gross  productivity  of  plants  and  animals  beyond  the  limits 
imposed  by  natural  systems.  Genetically  engineered  products  can  place  additional  burdens  on  natural 
systems.  For  example,  increased  photosynthesis  will  also  require  greater  use  of  soil  nutrients,  thus 
threatening  further  depletion  and  erosion  of  our  endangered  soil  base.  Arguing  in  this  fashion,  Rifkin 
presents  living  resources  “..as  finite  and  depletable  as  fossil  fuels.” 
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The  third  set  of  problems  stem  from  the  author’s  observation  that  transferring  genetic  traits  from  one 
species  into  the  permanent  hereditary  code  of  another  species  poses  grave  long-term  environmental  dangers 
and  raises  fundamental  moral  questions.  The  reader  will  wish  Rifkin  had  said  more  on  this  point,  surely  to 
be  heatedly  debated  in  the  decades  to  come. 

For  all  the  above  reasons,  the  author  finds  attempts  at  genetic  engineering  of  microbes,  plants  and 
animals  tantamount  to  playing  ^^ecological  roulette”.  He  asks  a  question  any  analyst  faced  with  the  use  of 
bioengineered  products  would  be  wise  to  ask.  “Are  we  wise  enough  and  smart  enough  to  begin  the  process 
of  redesigning  the  blueprints  of  living  systems  without  destroying  the  very  foundations  of  the  earth  s 
environment?” 
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A29:  “COMPARE:  An  Integrated  Tool  for  Hazard  Assessment  and  Risk  Analysis.”  By  G.  Rinaldi, 
S.  Stanghellini,  and  P,  Vestrucci.  Environmental  Software.  Vol.  7  No.  4. 1993.  pp.  203-215. 

Applicability 

COMPARE  is  a  highly  specialized  tool  developed  for  use  in  a  narrow  range  of  applications.  Though 
this  software  package  is  of  little  direct  use  or  interest  to  Corps  analysts,  it  is  interesting  because  it  illustrates 
that  integrated  risk  analysis  tools  have  been  developed  and  used.  MEPAS  is  another  software  tool  described 
in  one  of  the  articles  reviewed  in  this  annotated  bibliography  (see  Droppo  1993,  reference  A 15). 

COMPARE,  MEPAS  and  similar  tools  could  be  of  interest  to  modelers  and  Corps  analysts  who  have 
contractors  develop  risk  analysis  software  for  them.  This  software,  developed  by  Italian  scientists,  is 
available  from:  G.  Rinaldi,  NIER,  Via  S.  Stefano,  16  40125  Bologna  Italy.  Telephone:  39-51-239728,  fax: 
39-51-227824. 

Review 


COMPuter  Aided  Risk  Evaluation  (COMPARE)  is  a  prototype  integrated  tool  for  developing  hazard 
assessment  and  risk  evaluation  using  various  modeling  techniques,  scenario  simulations  and  spatial  analysis. 
It  is  a  geographical  information  system  (GIS)  that  integrates  data  preparation  and  input  modules,  modeling 
modules,  spatial  analysis  modules,  and  data  display  modules.  COMPARE  helps  analysts  address  industrial 
hazard  concerns  in  an  interdisciplinary  and  interactive  way. 

The  package  evaluates  the  consequences  and  risks  of  possible  accidents  in  an  industrial  area  through 
simulations.  The  user  interface,  data  structure,  and  system  architecture  are  described  with  flow-charts  and 
text,  but  of  necessity  remain  very  much  a  black  box  in  this  paper.  The  innovation  of  COMPARE  is  the  real 
integration  of  different  tools  and  techniques.  This  is  made  possible  by  a  computer  aided  design  (CAD) 
system  as  a  base  to  build  a  GIS,  based  on  a  suitable  “C-like”  language  able  to  deal  with  graphic  objects. 

COMPARE  is  used  for  industrial  accident  analysis.  It  would  seem  that  with  some  adaptation  a 
model  like  this  could  be  used  to  analyze  oil  spills  or  other  accident-like  events.  To  use  the  package  the 
analyst  must  first  digitize  the  plant/factory  and  geographical  maps.  The  program  provides  predefined  tanks, 
reactors,  furnaces,  heat  exchangers,  pipelines  and  towers.  New  components  can  also  be  defined.  The 
geographical  map  includes  three  territorial  entities:  vulnerability  centers  (schools  and  inhabited  areas,  for 
example);  resources  for  emergency  management  (e.g.,  hospitals,  fire  stations);  and  main  routes.  Data  are 
entered  for  each  entity  (distance  from  plant  and  population,  for  example). 

Next  the  analyst  must  locate  source  points  on  the  plant  map.  These  indicate  probability  of  accidents, 
substances  involved,  flow  rate,  temperature,  and  so  on.  The  analyst’s  most  important  task  is  the 
identification  of  possible  events  that  might  occur  at  the  plant.  Various  scenarios  associated  with  the  selected 


141 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography _ _ _ _ 

source  points  can  be  simulated  by  running  consequence  models  chosen  from  the  set  stored  in  the  system 
model  base.  The  output  of  the  evaluation  is  a  contour  map  of  the  consequences;  for  example,  radiation  for 
fire,  overpressure  for  explosion,  and  concentration  for  toxic  gas  dispersion. 

The  model  base  allows  the  simulation  of  different  accident  scenarios  including:  dispersion  of  neufral 
gas;  dispersion  of  heavy  gas;  unconfined  vapors  cloud  explosion;  jet  fire;  fire  ball,  and  pool  fire.  The  risk 
evaluation  estimates  the  expected  number  of  deaths  using  values  estimated  from  a  probit  model.  COMPARE 
has  been  used  for  the  development  of  documentation  for  a  chemical  plant  that  stores  and  handles  flammable 
and  toxic  substances. 
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A30:  “Risk  Sharing  in  the  Design  of  Environmental  Policy.”  By  Kathleen  Segerson.  American 

Agricultural  Economics  Association.  Vol.  68.  December  1986.  pp.  1261-1265. 

Applicability 

Who  should  bear  responsibility  for  damages  when  a  polluter  is  in  compliance  with  all  regulations 
but  still  causes  damages?  Is  it  the  polluter  because  he  has  caused  the  damage?  Or,  is  it  the  victim,  because 
the  polluter  was  in  compliance  with  society’s  regulations?  The  question  is  specific  but  not  narrow. 

This  article  is  more  academic  than  most  articles  in  this  review  in  that  its  focus  is  the  development 
of  theoretical  optimal  choices  under  varying  circumstances.  The  parallel  to  environmental  investments  might 
be  how  should  responsibility  for  stochastic  negative  outcomes,  analogous  to  pollution  events,  be  determined 
when  the  party  responsible  for  the  outcome,  the  polluter,  was  in  compliance  with  existing  regulations.  In 
other  words,  whose  fault  is  it  if  we  did  everything  we  were  supposed  to  have  done  but  something  still  went 
wrong?  More  importantly,  what  mix  of  regulation  and  judicial  system  resolution  should  be  used  to  achieve 
an  optimal  solution  to  problems  like  these?  This  article  answers  that  question  for  a  rather  specific  set  of 
circumstances.  It  answers  it  from  an  economic  perspective  assuming  the  expected  utility  of  the  polluter  and 
victim  are  the  values  to  be  optimized.  It  will  be  of  very  limited  applicability  to  most  environmental  resource 
investment  problems. 

Review 


The  article  begins  by  considering  two  policy  options  for  dealing  with  stochastic,  rather  than 
continual,  pollution  events.  One  is  through  the  ex  ante  constraints  of  the  regulatory  process.  The  other  is 
through  the  ex  post  constraints  of  the  judicial  system.  Which  policy  is  best?  Segerson  concludes  that 
depending  on  whether  polluters  and  victims  are  risk  averse  or  risk  neutral,  different  policies  may  be 
desirable.  The  allocation  of  risk  is  only  one  factor  to  be  considered.  Another  important  factor  is  the 
incentive  effect.  The  government’s  ability  to  induce  polluters  to  take  actions  to  reduce  their  probability  or 
magnitude  of  a  pollution  problem  depends  on  the  extent  to  which  pollution  events  are  observable  or 
unobservable.  Regulatory  models  generally  are  based  on  the  assumption  that  actions  by  the  polluter  to 
reduce  the  probability  or  magnitude  of  pollution  are  observable.  Incentive  scheme  models  are  based  on  the 
assumption  that  these  actions  are  not  observable. 

Thus,  Segerson  establishes  the  framework  for  examining  three  distinct  situations.  In  the  first,  there 
are  no  incentive  problems.  The  level  of  allowable  observable  and  unobservable  pollution  actions  can  be 
imposed  on  the  polluter.  The  problem  reduces  to  a  simple  maximization  of  the  expected  utilities  of  the 
polluter  and  victim  subject  to  the  constraint  of  a  minimum  level  of  utility  for  the  polluter.  This  is  a  simple 
risk  sharing  problem.  The  first  order  conditions  show  that  if  victims  are  risk  averse  and  polluters  are  risk 
neutral,  polluters  should  be  subject  to  full  ex  post  damages.  If  the  opposite  is  true  and  victims  are  risk 
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neutral  and  polluters  are  risk  averse,  then  there  should  be  no  ex  post  liability.  If  both  parties  are  risk  averse, 
a  partial  ex  post  liability  scheme  is  optimal. 

In  the  second  application  of  the  model  the  firm  must  be  induced  to  take  socially  efficient  actions. 
In  the  first  model  these  actions  could  be  Imposed.  The  results  show  that  if  polluters  are  risk  averse  and  their 
actions  affect  the  probabilities  of  alternative  states  of  nature,  then  a  partial  liability  scheme,  somewhere 
between  full  ex  post  and  full  ex  ante  liability,  is  efficient. 

In  the  third  model,  observable  actions  are  subject  to  regulation  and  unobservable  actions  are  subject 
to  incentives.  In  this  joint  use  of  regulation  and  liability  the  author  finds  that  if  the  polluter  s  unobserved 
actions  affect  states  of  nature,  some  positive  level  of  ex  post  liability  is  optimal.  If,  in  addition,  polluters  are 
risk  averse,  a  system  of  regulation  and  partial  ex  post  liability  is  preferred. 

The  principal  finding  is  that  a  policy  that  couples  regulation  of  observable  actions  with  a  system  of 
partial  liability  seems  to  be  a  preferable  way  to  balance  the  need  for  risk  sharing  and  incentives.  This  article 
will  not  be  easy  for  most  readers  to  follow.  The  arguments  are  rather  straightforward  for  economics 
graduates  students  but  they  skip  far  too  many  steps  for  the  non-economist  to  follow.  The  other  significant 
limitation  is  finding  relevance  for  the  conclusions  in  the  environmental  investment  field. 
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A31:  “Environmental  Risk  Analysis:  An  Overview.”  By  M.P.  Singh  and  Curtis  C.  Travis.  Risk 
Analysis:  An  Official  Publication  of  the  Society  for  Risk  Analysis.  Vol.  11.  No.  3.  September 
1991.  pp.  377-379. 

Applicability 

This  is  a  paper  for  and  about  India,  responding  in  part  to  the  need  to  manage  environmental  risks 
made  obvious  by  the  Bhopal  incident.  This  article  was  reviewed  in  an  effort  to  avoid  overlooking 
international  initiatives  in  the  area  of  environmental  risk  analysis.  Indian  risk  assessments,  as  described  in 
this  paper,  are  somewhat  behind  assessments  in  the  U.S.  However,  they  have  learned  from  our  experience. 
For  the  most  part  this  article  is  like  an  early  1980s  rehash  of  the  state-of-the-art  of  risk  analysis.  There  is 
little  of  real  use  in  this  article.  It  does,  however,  provide  examples  of  cultural  adaptations  of  U.S.  analytical 
techniques. 

Review 


The  Government  of  India’s  Ministry  of  Environment  and  Forests  sponsored  a  workshop  on  risk 
assessment  techniques  and  management  in  October  1987.  The  need  to  control  emissions  of  hazardous  air 
pollutants,  plan  for  the  prevention  of  accidental  releases  of  hazard  substances  into  the  environment,  and  to 
develop  appropriate  emergency  responses  was  recognized  by  all  present.  Recommendations  from  that 
workshop  would,  presumably,  guide  the  development  of  the  nation’s  environmental  risk  analysis  program. 

A  comprehensive  program  for  assessment  of  risk  due  to  hazardous  air  pollutants  was  identified  as 
a  primary  need.  Though  prevention  was  recognized  as  the  best  means  for  protecting  human  health  and  the 
environment,  emergency  response  plans  would  be  needed.  Local  emergency  planning  committees  would 
conduct  site-specific  hazard  analyses  to  develop  emergency  plans.  Technical  guidance  for  identifying  and 
analyzing  hazards  must  be  provided.  Risk  liability  insurance  was  identified  as  an  avenue  deserving  future 
investigation  as  part  of  emergency  planning.  The  emergency  preparedness  initiatives  included  the  same 
elements  you  would  expect  to  find  in  a  write-up  of  an  American  plan.  This  raises  the  question  in  our  minds 
whether  something  with  such  psychological/sociological  roots  can  be  transplanted  so  readily  from  Western 
to  Eastern  cultures  without  substantial  retooling.  This  concern  was  not  reflected  in  the  paper,  however. 

Land-use  planning  was  another  major  topic  of  discussion.  The  major  thrust  was  to  keep  population 
centers  away  from  potentially  hazardous  facilities.  A  three-tiered  buffer  zone  has  been  suggested  for  India. 
The  inner  zone  is  the  area  of  greatest  risk.  It  should  remain  an  unpopulated  green  belt.  The  middle  zone  has 
moderate  risks  but  population  should  still  be  prohibited.  The  outer  zone  has  low  risks  but  sensitive 
development  such  as  hospitals,  schools,  and  the  like  will  need  case-by-case  consideration. 

The  workshop  participants  requested  a  database  information  system  be  developed.  It  would  include 
toxicological  data  needed  to  develop  ambient  standards  as  well  as  a  list  of  symptoms,  antidotes,  and  methods 
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of  treatment  for  exposure  to  hazardous  materials.  Environmental  data  and  an  inventory  of  hazardous 
materials  would  also  be  included  in  the  database.  Finally,  training  and  public  awareness  were  recognized 
as  significant  needs  for  the  country. 
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A32;  “Ecology  and  Design:  An  Introduction.”  By  Ian  R.  Smith.  Journal  of  Environmental 
Management.  Vol.  26.  No.  2.  March  1988.  pp.  103-109. 

Applicability 

The  author  in  this  piece  argues  that  applying  design  principles  to  ecological  problems  is  one 
approach  to  studying  these  problems.  One  of  the  more  important  issues  the  Corps  will  face  in  its  transition 
to  adaptive  management  for  restoration  of  environmental  resources  is  understanding  how  ecological  systems 
and  environmental  investment  projects  function  and  how  they  fail.  This  article  offers  an  alternative  for 
treating  applied  ecology  as  a  problem  in  design.  Unfortunately,  this  good  idea  is  not  as  well  developed  as 
many  readers  might  like.  Nonetheless  it  provides  a  focused  idea  that  others  may  be  able  to  develop  further. 

Review 


Although  the  science  of  ecology  is  developing  we  do  not  yet  have  a  methodology  for  tackling  applied 
problems  systematically.  In  this  paper  Smith  explores  the  possibility  of  approaching  problems  in  applied 
ecology  as  design  problems.  He  considers  and  rejects  some  definitions  of  design  in  favor  of  his  own.  Smith 
defines  design  as  “..the  deliberate  creation  of  an  object  and/or  procedure  in  order  to  fulfill  some  specified 
function.”  The  author  then  adopts  the  seven  interacting  design  criteria  of  Papanek  as  the  basis  for  his  article. 
These  criteria  are:  1)  function  (is  it  clear  what  the  design  is  expected  to  do?);  2)  use  (does  it  work?);  3)  need 
(is  it  necessary?);  4)  telesis  (is  it  appropriate  to  the  society  and  conditions  in  which  it  has  to  function?);  5) 
association  (is  it  psychologically  acceptable,  i.e.,  are  we  comfortable  with  it?);  6)  aesthetics  (is  it  visually 
exciting,  a  source  of  delight?);  and  7)  method  (how  is  it  to  be  made?). 

Smith  concentrates  on  the  function  and  use  criteria  of  ecological  design.  Telesis,  association  and 
aesthetics  are  normally  within  the  purview  of  the  landscape  architect.  Need  frequently  is  determined  by 
benefit-cost  analysis.  Method,  he  argues,  merits  a  separate  discussion  at  another  time  and  place. 

Just  what  is  it  that  is  being  designed?  The  unit  of  design  can  be  defined  as  any  number  of  things,  so 
the  starting  point  for  any  analysis  should  be  a  clear  understanding  of  what  the  unit  under  consideration  is. 
It  could  be  a  few  acres  of  woodland  or  the  whole  of  the  British  Isles. 

The  function  of  an  ecological  unit  must  be  explicitly  described.  Smith  indicates  there  may  be  many 
functions,  including  technical,  perceptual,  conservation  and  ecological.  Technical  functions  like  water 
supply,  recreation,  agriculture  and  the  like  are  the  easiest  to  identify.  Ecological  functions  are  frequently 
defined  as  preservation,  maintenance  and  display  of  species  and  habitats.  Smith  points  out  that  the  function 
of  ecological  projects  are  not  often  identified,  perhaps  in  the  belief  that  a  clear  statement  of  function  provides 
opponents  with  easy  ammunition.  Once  a  function  is  established  as  unacceptable  few  other  arguments  will 
carry  any  weight. 


147 


Incorporating  Risk  and  Uncertainty  into  Environmental  Evaluation: 

An  Annotated  Bibliography _ 

“Use”  gets  at  the  question,  does  it  work  or,  more  appropriately,  what  is  the  probability  it  will  not 
work?  Once  the  function  has  been  established  it  is  then  possible  to  identify  the  modes  of  failure,  that  is,  the 
ways  in  which  the  project  fails  to  function.  Smith  uses  a  simple  load/factor  of  safety  analysis  approach  to 
the  issue  of  failure  with  reliability,  recovery  rate,  and  vulnerability  as  the  measures  of  system  performance. 
He  offers  an  example  from  the  literature  (Holling  1973)  to  illustrate  dynamic  system  performance  in  a  simple 
way.  When  the  state  of  the  system  of  interest  falls  beneath  a  failure  threshold,  a  problem  exists.  The  amount 
of  time  the  system  state  remains  below  this  threshold  determines  the  system’s  resilience.  The  problem  Smith 
acknowledges  is  determining  what  measure  of  a  system  state  could  be  used  to  determine  the  failure  threshold 
for  an  ecological  system. 

Local  extinction  of  a  species,  changes  in  energy,  conversion  efficiency,  and  alterations  of  community 
structure  and  species  diversity  are  offered  as  examples  of  possible  thresholds.  Smith  never  really  pulls  his 
argument  together  effectively.  We  think  it  might  be:  if  you  can  define  an  ecological  unit/system  in  explicit 
terms,  identify  its  functions,  and  the  uses  to  which  it  is  put,  it  is  then  possible  to  identify  modes  of  failure. 
Smith  does  say  that  “..it  will  never  be  possible  to  make  decisions  solely  on  the  basis  of  the  implications  of 
the  mathematical  analysis  of  ecosystem  behavior.”  Combining  common  sense  and  experience  to  understand 
the  possible  modes  of  failure  and  the  requirements  to  avoid  such  failures  is  the  true  value  of  the  attempt  to 
apply  design  principles  to  applied  ecological  problems. 
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A33:  “Risk  Analysis  for  the  Concho  Water  Snake.”  By  Michael  E.  Soule.  Endangered  Species 

Update.  Vol.  6.  No.  10.  August  1989.  pp.  19-25. 

Applicability 

This  is  the  kind  of  article  we  had  hoped  to  find  hundreds  of  when  this  literature  review  was  being 
planned.  It  is  well-written  and  accessible  to  all.  It  deals  directly  and  pragmatically  with  a  well-defined 
problem  and  uses  techniques  that  are  easy  to  follow.  It  provides  a  clear  model  for  environmental  resource 
planners  to  model. 

That  is  the  good  news.  The  bad  news  is  that  it  is  the  only  article  with  all  these  qualities  that  we 
found.  It  is  highly  recommended,  however,  as  an  example  of  how  analysts  can  approach  recovery  planning 
where  they  have  to  make  many  decisions  without  adequate  information.  It  relies  on  subjective  risk 
assessments  and  the  use  of  decision  trees,  two  useful  tools  for  the  Corps’  risk  assessment  of  environmental 
investments.  Estimating  the  probabilities  of  extinctions  of  species,  at  least  in  the  vicinity  of  a  project,  offers 
a  great  deal  of  promise  as  an  assessment  measure  for  environmental  resource  planning.  This  article  provides 
an  exceptional  introduction  to  the  use  of  this  concept. 

Though  we  are  not  particularly  impressed  with  the  subjective  assessment  done  for  this  article,  the 
article  itself  is  an  excellent  example  of  how  one  might  effectively  describe  an  analysis  in  writing.  There  is 
much  in  this  article  worth  using  and  emulating. 

Review 


Nerodia  harteri  paucimaculata,  the  concho  water  snake,  lives  in  a  small  area  of  central  Texas. 
Reliable  estimates  of  its  population  size  are  lacking.  Their  young  feed  in  rocky  shallows  or  riffles.  Its  other 
habitat  requirements  include  basking  sites,  hibemacula,  and  pools  for  adult  feeding.  In  a  deft  manner  the 
author,  in  three  short  paragraphs,  makes  you  feel  like  you  know  this  snake. 

The  motivation  for  this  article  was  the  construction  of  Stacy  dam  at  the  confluence  of  the  Colorado 
and  Concho  Rivers,  a  site  that  includes  much  prime  habitat  for  as  much  as  one-fourth  to  one-half  the  species. 
The  management  goal  is  “..to  maintain  the  viability  of  the  species  in  the  vicinity  of  its  current  geographic 
range.”  In  this  analysis  the  risk  of  extinction  was  assessed  over  the  next  25  years.  Such  clear  objectives 
provide  a  good  example  for  Corps  planners. 

The  specific  objectives  of  this  study  were  to:  1 )  identify  potential  threats,  now  and  in  the  future,  to 
the  snake;  2)  identify  and  clarify  the  various  management  interventions  available  for  the  snake;  3)  estimate 
the  probabilities  that  each  of  these  threats  will  occur  and  that  the  management  measures  will  be  effective; 
4)  estimate  the  probability  of  extinction  of  the  snake,  given  various  combinations  of  events  and  management 
measures;  and,  5)  analyze  the  sensitivity  of  the  various  estimates  to  changes  in  their  values.  The  analysis 
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is  intended  to  provide  the  best  available  information  about  the  relative  risks  of  different  management 
measures. 

Soule  provides  a  clear  and  well  explained  example  of  how  such  objectives  can  be  pursued  through 
decision  analysis  that  relies  on  subjective  probability  assessments  and  decision  trees.  The  analysis  showed 
that  the  “no  dam”  alternative  was  best  for  the  preservation  of  the  snake.  Even  this  alternative  carried  an  1 1% 
chance  of  extinction  over  the  next  25  years,  however.  This  alternative  was  followed  from  best  to  worst  by 
the  following:  1)  artificial  river  (29%  chance  of  extinction)  and  flushing  and  scouring  (also  29%);  3) 
artificial  riffles  (30%  chance);  4)  reservoir  substrata  (38.5%);  5)  translocation  and  reintroduction  (39%);  and, 
6)  no  intervention  (40%  chance  of  species  extinction). 

In  a  sensitivity  analysis  the  author  attempts  not  to  find  more  accurate  estimates  of  these  probabilities 
but  to  determine  what  values  are  more  critical  to  the  analysis.  The  author’s  conclusions  are,  like  the  rest  of 
his  work,  clear  and  effective  ways  to  deal  with  the  inherent  uncertainties  of  recovery  work.  “The  consensus 
of  the  experts  appears  to  be  that  none  of  the  interventions  independently  reduces  the  probability  of  extinction 
by  more  than  10  to  15  percent  below  the  40  percent  ”no  intervention”  figure...  It  is  conceivable  that  the 
interventions  examined  herein,  given  that  they  all  succeed,  would  reduce  the  extinction  vulnerability  of  the 
snake  to  a  level  below  what  it  is  today.  The  probability  of  such  a  decrease  in  jeopardy  cannot  be  ascertained 
given  all  of  the  uncertainty  surrounding  the  situation  at  present...  it  is  quite  likely  that  the  estimates  of  the 
probability  of  extinction  would  be  substantially  higher  if  the  temporal  window  were  50  or  100  years.” 
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A34:  “Ecological  Risk  Assessment:  Its  Role  in  Risk  Management.”  By  David  James  Stout  and 
Robin  A.  Streeter.  The  Environmental  Professional.  Vol.  14, 1992.  pp.  197-203. 

Applicability 

As  is  evident  from  several  other  reviews  in  this  report,  ecological  risk  assessment  is  an  evolving  field 
of  endeavor.  Initial  efforts  to  conduct  ecological  risk  assessments  have  arisen  from  the  need  for  site 
remediation.  Initial  efforts  have  focused  on  imitating  and  attempting  to  adapt  the  environmental  risk 
assessment  paradigm  of  the  EPA. 

This  article  provides  a  succinct,  albeit  brief,  summary  of  the  EPA’s  ecological  risk  assessment 
efforts.  This  area  of  analysis  is  going  to  be  important  to  the  Corps  for  any  potential  Superfund  (CERCLA) 
work,  for  dredging/sediment  disposal,  and  any  other  restoration  work  involving  toxic  wastes.  It  is  useful 
for  Corps  analysts  involved  in  environmental  resource  risk  analysis  to  become  aware  of  the  environmental 
and  ecological  risk  assessments  methods  currently  in  use  by  other  agencies.  This  article  provides  a  nice 
introductory  summary  of  EPA’ s  efforts  in  ecological  risk  assessment. 

Review 


Risk  assessment  is  critical  to  decision-making  concerning  site  remediation  and  has  been  for  years. 
While  the  focus  in  the  early  years  was  on  human  health,  there  is  now  a  need  for  a  reasonable  and  consistent 
approach  to  evaluating  ecological  risk.  With  this  as  their  starting  point,  the  authors  provide  a  brief  but 
effective  summary  of  EPA’s  risk  assessment  methodology. 

Developing  conclusions  about  ecological  risks  has  been  frustrated  by  the  following:  1)  limitations 
in  available  ecological  data;  2)  paucity  of  ecotoxicity  benchmarks;  3)  uncertainty  about  ecosystem  response 
to  chemical  perturbation;  and  4)  lack  of  Superfund  policy  defining  significant  ecological  risk.  Noting  that 
numerous  methods  for  conducting  ecological  risk  assessments  have  been  proposed  and  reviewed,  the  authors 
present  a  case  study  of  an  ecological  risk  assessment  for  the  characterization  of  sediments  in  the  Great  Lakes. 
The  ecological  risk  assessment  process  consisted  of  four  interrelated  steps:  1)  identify  hazards  through  the 
selection  of  chemicals  of  concern  and  characterize  baseline  ecology  and  potential  receptors;  2)  evaluate 
potential  exposure;  3)  assess  chemical-  and  media-specific  ecotoxicity;  and,  4)  characterize  potential  risks. 
Each  of  these  steps  is  briefly  described. 

The  case  study  offers  a  simple  equation  for  sediment  concentration  that  is  used  as  a  screening  level 
criteria.  Sediment  concentrations  derived  using  the  EPA  approach  were  compared  to  sediment  levels  of 
chemicals  of  concern  to  develop  a  hazard  ratio.  Ratios  less  than  unity  indicate  no  likely  adverse  effects  to 
biota.  The  preliminary  conclusions  in  the  case  study  were:  1)  acute  toxicity  to  aquatic  life  inhabiting  the 
lake  most  likely  is  not  a  concern  based  on  either  measured  or  predicted  exposure  concentrations;  2)  mercury 
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probably  represents  the  primary  chemical  of  concern;  and,  3)  biomagnification  potential  exists  for  both 
mercury  compounds  and  chlorinated  benzenes. 

Stout  and  Streeter  identify  several  issues  to  be  addressed  in  the  development  of  an  ecological  risk 
assessment  program.  These  include  the  need  to  establish  risk  management  objectives  and  goals  if  we  are 
to  protect  “...environmental  receptors  beyond  humans”. 
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A35:  Ecological  Risk  Assessment.  By  Glenn  W.  Suter  II,  Lawrence  W.  Barnthouse,  Steven  M. 
Bartell,  Theodore  Mill,  Donald  Mackay,  and  Sally  Paterson.  Boca  Raton:  Lewis  Publishers, 
1993.  pp.  xvii,  538.  Four  parts  and  thirteen  chapters.  ISBN  0-87371-875-5. 

Applicability 

Lewis  Publishers  seems  to  have  carved  a  niche  for  itself  as  the  publisher  of  some  of  the  most  up-to- 
date  and  complete  texts  on  ecological  and  environmental  risk  assessment.  In  this  book  edited  by  Suter,  also 
the  principle  author,  advanced  students  in  environmental  sciences  and  practicing  environmental  scientists 
will  find  a  good  introduction  to  risk  analysis  as  it  is  practiced  by  the  EPA  in  the  area  of  human  health  effects. 
The  book  assumes  a  basic  knowledge  of  ecology,  toxicology,  chemistry,  mathematics  and  statistics.  It 
assumes  no  knowledge  of  risk  analysis.  It  is  an  excellent  book  but  it  does  at  times  require  a  rather  diverse 
vocabulary  in  the  natural  sciences.  It  is  recommended  to  anyone  interested  in  a  fuller  understanding  of  the 
field  that  has  come  to  be  known  as  ecological  risk  assessment. 

Review 


The  book  consists  of  four  parts.  The  first  part  (three  chapters)  is  devoted  to  an  introduction  to 
ecological  risk  assessment.  The  second  part  (three  chapters)  addresses  exposure  assessments  for  predictive 
risk  assessments.  Part  Three  (also  three  chapters)  focuses  on  effects  assessment.  The  last  part  discusses 
unconventional  ecological  risk  assessment  (four  chapters).  A  handy  glossary  is  also  Included. 

Chapter  1  defines  ecological  risk  assessment  as  the  estimation  of  risks  posed  by  chemicals. 
Environmental  risk  assessment  is  described  as  a  far  more  encompassing  field  of  endeavor.  Chapter  2  defines 
a  number  of  concepts  essential  to  ecological  risk  assessment.  ’The  purpose  of  ecological  risk  assessment 
is  to  contribute  to  the  protection  and  management  of  the  environment  through  scientifically  credible 
evaluation  of  the  ecological  effects  of  human  activities"  p.  (21).  Endpoints,  essential  to  any  assessment,  are 
defined  at  length.  In  brief,  an  endpoint  is  a  formal  expression  of  the  environmental  value  to  be  protected. 
Assessment  methods,  including  physical,  statistical,  and  mechanistic  models,  are  discussed.  The  requisite 
discussion  of  probability  and  risk  and  uncertainty  can  also  be  found,  but  they  are  treated  more  thoroughly 
in  other  publications  covered  in  this  review. 

The  basic  ecological  risk  assessment  paradigm  is  presented  in  Chapter  Three.  It  is  based  on  the 
human  health  risk  assessment  paradigm,  and,  though  there  are  many  similarities,  there  are  substantial 
differences  as  well.  Step  one  is  to  define  the  hazard.  This  requires  defining  endpoints,  describing  the 
environment,  and  obtaining  source  items.  Step  two  is  the  exposure  assessment  (the  dose-response 
assessment  in  the  human  health  paradigm).  Step  three  is  the  effects  assessment.  Step  four  is  the  risk 
characterization.  The  authors  add  risk  management  as  an  explicit  fifth  step  that  is  often  assumed  to  be  part 
of  the  fourth  step  in  other  discussions  of  the  paradigm. 
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In  Part  Two  the  chapters  are  devoted  to  a  discussion  of  methods  for  estimating  the  exposure  of 
organisms  to  chemical  pollutants.  Chapter  Four  on  environmental  chemistry  deals  with  estimating  the 
physical/chemical  properties  of  pollutants  to  determine  their  behavior  in  the  environment.  Billions  of  tons 
of  natural  and  synthetic  organic  chemicals  move  through  the  air,  water  and  soil  environments  each  year. 
Environmental  chemistry  attempts  to  describe  and  quantify  environmental  processes  and  the  rates  and 
products  of  movement  and  transformation.  This  chapter  focuses  on  environmental  reactions  to  organic 
compounds  by  presenting  some  of  the  underlying  principles  of  environmental  modeling. 

Chemical  fate  and  transport  models  are  discussed  in  Chapter  Five.  These  models  take  data 
concerning  chemical  discharge  rates  and  concentrations  together  with  information  about  the  state  of  the 
environment  and  estimate  the  amounts  and  concentrations  of  chemicals  present  in  each  medium.  It  also 
estimates  rates  at  which  the  chemicals  will  degrade  and  be  transported  from  place-to-place.  This  analysis 
provides  an  estimate  of  how  long  the  chemical  will  reside  in  the  environment. 

The  quantitative  description  of  the  system  is  frequently  referred  to  as  a  "mass  balance"  model.  The 
mass  balance  model  example  is  carried  through  much  of  the  chapter  that  ends  with  a  description  of  other 
environmental  models  of  chemical  fate  (fugacity  models,  GEOTOX,  Environmental  Partitioning  Models, 
and  others). 

Chapter  Six  addresses  exposure,  defined  as  contact  with  a  chemical  or  physical  agent.  First,  the 
chapter  considers  the  relevance  of  the  concentrations  in  media  (fluids,  solids,  multiple  media)  predicted  by 
the  fate  models  to  the  environment  as  experienced  by  organisms.  Second,  it  considers  how  the  behavior  of 
organisms  results  in  the  uptake  of  pollutants.  Third,  using  toxicokinetic  models,  it  discusses  how  a 
concentration  of  a  substance  in  a  medium  can  be  converted  into  an  exposure  level. 

Part  Three,  Effects  Assessments,  discusses  methods  for  estimating  the  ecological  effects  that  result 
from  exposure  to  pollutant  chemicals.  In  Chapter  Seven  organism-level  effects  are  covered.  Tests  for  effects 
and  models  for  effects  are  each  addressed  in  turn.  The  sensitivity  of  endpoints  to  differences  in  species,  test 
conditions,  exposure  duration  and  other  variables  is  also  discussed.  Numerous  examples  are  presented.  The 
chapter  also  considers  models  of  chemical  mixtures.  Probit  models  are  used  for  estimation  in  some  cases. 
Results  are  presented  in  the  form  of  isobolograms.  Finally,  the  risks  to  individual  organisms  is  estimated 
or  “characterized”,  in  the  vernacular  of  the  ecological  risk  assessment. 

Chapter  Eight  focuses  on  population-level  effects.  Endpoints  for  populations  tend  toward  total 
population  density  or  biomass,  age  or  size  distribution,  or  yield  available  to  harvest.  Population  analysis 
receives  an  abbreviated  discussion  after  which  examples  of  specific  risks  of  chemicals  to  specific  species 
are  presented.  Linking  toxicity  data  to  population  models  using  the  models  of  this  chapter  requires  the 
development  of  exposure-response  functions  that  deal  with  uncertainty  of  varying  types. 

The  last  chapter  of  this  section  presents  ecosystem-level  effects  assessments.  Frequently  the 
endpoints  will  have  ecosystem  properties.  At  least  four  types  of  effects  can  occur  at  the  ecosystem-level  that 
do  not  occur  at  the  organism  or  population  level.  First,  effects  on  a  population's  ability  to  interact  with 
populations  of  other  species  may  occur.  Second,  indirect  effects  on  a  population  due  to  effects  on  the 
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populations  with  which  it  interacts  can  occur.  Third,  there  can  be  structural  changes  in  the  properties  of  an 
ecosystem.  Fourth,  there  can  be  changes  in  the  functional  properties  of  an  ecosystem.  Numerous  tests  and 
models  for  ecosystem  effects  are  discussed. 

In  the  last  part  of  the  book,  unconventional  ecological  risk  assessments  are  discussed.  Risks  that  do 
not  fit  the  paradigm  of  the  earlier  text  are  presented  here.  Chapter  Ten  on  Retrospective  Risk  Assessment 
may  be  of  interest  to  Corps  analysts.  While  predictive  risk  assessment  was  the  focus  of  the  early  text,  this 
chapter  acknowledges  that  assessment  of  pollution  effects  that  began  in  the  past  and  may  have  ongoing 
consequences  (such  as  waste  sites)  are  becoming  increasingly  important.  Such  retrospective  assessments 
are  more  diverse  than  the  predictive  models  addressed  up  to  this  point  in  the  book.  While  predictive  models 
have  a  proposed  source,  the  impetus  for  retrospective  models  can  be  a  source,  observed  effects,  or  evidence- 
of  exposure.  Each  type  of  impetus  is  discussed  in  turn.  Examples  are  provided  for  acid  deposition  and 
fisheries,  ecological  effects  of  a  waste  site,  and  effects  of  granular  carbofuran  on  birds. 

Regional  risk  assessments  are  presented  in  Chapter  Eleven.  These  assessment  address  larger  scale 
effects  of  multiple  actions  (e.g.,  Exxon  Valdez,  Chernobyl).  In  Chapter  Twelve  the  authors  argue  that  much 
of  environmental  science  has  been  devoted  to  monitoring  the  environment  for  purposes  of  surveillance  rather 
than  to  support  risk  assessments.  Surveillance  monitoring  serves  the  purposes  of:  assessing  regulatory 
compliance;  as  a  means  of  system  control;  assessing  environmental  quality;  and  detecting  the  unexpected. 
Surveillance  describes  things  as  they  happen  rather  than  predicting  or  explaining  why  they  happened,  as 
predictive  or  retrospective  risk  analysis  would  demand.  However,  the  authors  argue  there  is  no  reason  why 
surveillance  can't  serve  risk  assessments  as  well. 

Suter  begins  the  chapter  on  exotic  organisms  with  an  admonition  from  Lewis  Carroll  to  beware  the 
Jabberwock,  Jubjub  bird,  and  Bandersnatch.  This  chapter  addresses  the  introduction  of  exotic  organisms  to 
new  areas.  A  predictive  risk  methodology  based  on  the  same  paradigm  is  offered  in  this  chapter  that  extends 
the  ideas  presented  in  the  Caltagirone  (1980)  article  reviewed  elsewhere  in  this  report  (reference  A9).  The 
current  author  finds  the  risk  somewhat  more  imposing.  Five  typical  endpoints  for  such  actions  are  identified. 
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A36:  “The  Complementarity  Between  Environmental  and  Economic  Risk:  An  Empirical  Analysis.” 
By  Paul  H.  Templet  and  Stephen  Farber.  Ecological  Economics.  Vol.  9.  No.  2.  February  1994. 
pp.  153-165. 

Applicability 

This  was  another  article  with  a  very  promising  title  that  didn’t  deliver  what  we  had  hoped  for.  The 
supposed  conflict  between  economics  and  the  environment  is  the  focus  for  this  paper.  It  makes  some  very 
good  points,  but  it  does  not  deal  with  risk  in  a  very  explicit  way.  The  risk  is  more  implicit.  Though  we  re 
not  sure  the  authors  would  describe  it  this  way,  their  implicit  risk  argument  goes  as  follows.  Economic 
development,  i.e.,  jobs,  bring  risks,  i.e.,  pollution.  If  we  lower  the  risk  of  economic  ruin,  i.e,  increase  jobs, 
we  increase  the  risk  of  environmental  ruin.  Lower  the  risk  of  environmental  ruin,  i.e.,  pollution,  and  we 
could  increase  the  risk  of  economic  ruin. 

The  authors  deserve  points  for  their  creativity.  They  are  attempting  to  integrate  economic  and 
environmental  metrics  in  a  way  that  might  help  decision  makers  think  about  these  risks.  It  s  a  first  attempt 
in  the  new  field  of  ecological  economics.  As  such  it  is  welcome;  but  it  is  also  less  than  compelling. 

Review 


The  authors  begin  from  a  commonly  assumed  premise.  They  state,  “The  traditional  view  is  that 
enhancements  to  environmental  conditions  are  costly  to  economic  development.  In  other  words, 
environmental  and  economic  risks  are  inversely  related.”  From  this  position  they  present  a  position  they 
consider  more  realistic:  “...the  loss  of  an  ecosystem  base,  or  increased  environmental  risk  and  reduced 
economic  carrying  capacity  of  the  ecosystem,  reduces  the  long-term  economic  welfare  secured  from  the 
ecosystem.”  In  other  words,  economics  and  ecology  are  not  contraposing  forces,  they  are  complementary. 

This  study  examines  the  long-term  complementarity  of  this  relationship  by  examining  the  ratio  of 
toxic  chemical  emissions  to  jobs  (E/J)  to  see  if  relative  risks,  as  reflected  by  this  ratio,  are  consistently 
related  to  traditional  welfare  measures  such  as  income  and  employment  across  the  U.S.  The  authors  posit 
that  economic  and  environmental  development  are  the  result  of  a  combination  of  comparative  advantages 
in  economic  and  environmental  resources  as  well  as  public  policies. 

Using  data  from  EPA’s  Toxic  Release  Inventory  (TRI)  and  the  Statistical  Abstract  of  the  U.S.,  the 
author’s  estimated  the  E/J  for  various  SIC  code  manufacturing  sectors.  This  ratio  is  represented  by  the 
authors  as  a  measure  of  environmental  risk.  The  E/J  reflects  total  TRI  emissions  per  manufacturing  sector 
job.  The  ratios  ranged  from  a  low  for  apparel  of  2  to  a  high  for  chemicals  of  3,237.  In  addition,  ratios  were 
calculated  for  all  fifty  states.  The  variations  within  and  across  industries  and  states  are  discussed.  Simple 
correlations  of  the  E/J  to  economic  variables  were  calculated.  The  E/J  and  per  capita  income  are  negatively 
related,  while  the  E/J  is  positively  related  to  unemployment  and  poverty.  The  authors  provide  numerous 
other  correlations  using  different  measures  of  environmental  risk  including  ratios  of  hazardous  wastes  to 
jobs,  total  Btu  use  to  jobs,  and  the  Green  Index. 
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The  analysis  seems  somewhat  superficial  at  times.  The  authors  do  not  present  a  convincing  case  that 
E/J  or  any  of  the  other  measures  is  a  meaningful  metric  of  environmental  risk,  a  term  they  fail  to  adequately 
define.  It  is  almost  as  if  the  authors  found  statewide  data,  developed  an  index  and  then  began  to  look  for 
things  to  correlate  it  with.  The  major  weakness  of  this  paper  is  that  it  lacks  a  theoretical  framework.  The 
reader  never  gets  a  good  feel  for  why  the  authors  are  correlating  these  variables.  Why  these  variables?  What 
does  it  mean?  They  fail  to  explain  the  significance  of  any  of  their  relationships,  perhaps,  we  fear,  because 
there  is  none.  They  seem  to  implicitly  define  economic  risk  as  poor  economic  conditions  and  they  conclude 
there  is  evidence  that  environmental  risk  and  economic  risk  are  complementary.  What  is  missing  from  this 
article  is  that  there  is  no  cause  and  effect  theory,  nor  cause  and  effect  analysis.  In  fairness  to  the  authors, 
they  acknowledge  this  near  the  end  of  the  article. 

The  paper  describes  potential  policy  uses  of  the  E/J  ratio.  These  include:  reemphasizing  Jobs, 
institutional  structures  for  improving  the  E/J,  development  transition,  and  using  the  E/J  as  an  environmental 
scorecard.  There  was  a  one  year  trial  use  of  the  E/J  as  an  environmental  scorecard  in  Louisiana  in  1990.  The 
scorecard  was  a  ranking  system  that  applied  environmental  and  economic  criteria  to  a  company’s  tax 
exemption  status.  In  an  innovative  but  amazingly  complex  and  arbitrary  fashion,  companies  requesting  tax 
exemptions  were  rated  using  some  of  the  techniques  described  in  this  paper.  The  program  was  abandoned 
after  one  year. 

Though  we  have  been  somewhat  critical  of  some  aspects  of  this  paper  it  makes  an  important  and 
significant  contribution  to  the  environmental  risk  analysis  literature  that  should  be  of  interest  to  Corps 
personnel.  It  tried  something  different,  something  new.  This  is  not  another  risk-benefit  analysis.  It  is  not 
a  benefit-cost  ratio.  It  was  an  attempt  to  develop  new  metrics  that  reflect  not  just  economics  and  not  just 
environmental  values.  It  serves  as  an  example  of  innovative  thinking  as  we  investigate  uncharted  territory 
in  environmental  resource  risk  analysis. 
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LITERATURE  SOURCES  TO  MONITOR 


INTRODUCTION 

The  risk  and  uncertainty  literature  is  relatively  recent.  The  amount  of  risk  material  written  about 
environmental  planning  in  general  and  environmental  restoration  in  particular  is,  unfortunately,  minuscule. 
Over  time,  risk  analysis  is  being  used  in  more  and  more  specific  applications.  Gradually,  it  will  be  applied 
to  environmental  issues  that  transcend  the  human  health  focus  the  environmental  risk  analysis  literature  has 
had  to  this  point  in  time.  For  example,  at  this  writing,  IWR  Report  96-R-8,  An  Introduction  to  Risk  and 
Uncertainty  in  the  Evaluation  of  Environmental  Investments,  contains  an  example  of  a  risk-based  analysis 
of  a  habitat  suitability  index  model.  The  Corps’  continuing  interest  in  improving  decisions  through  risk 
analysis  alone  insures  that  there  will  be  more  applications  of  risk  analysis  to  environmental  studies. 

Another  significant  recent  development  has  been  USEPA’s  publication  of  Draft  Proposal  Guidelines 
for  Ecological  Risk  Assessment,  EPA/630/R-95/002  October  1995,  External  Review  Draft.  A  review  of  this 
document  has  not  been  provided  here  because  EPA  has  requested  that  the  report  be  neither  quoted,  cited,  or 
distributed  at  this  point.  Nonetheless,  the  report  is  expected  to  lead  to  a  final  set  of  guidelines  at  some  point 
in  the  future.  The  ecological  risk  assessment  framework  offers  considerable  promise  to  the  extension  of  risk 
analysis  into  the  field  of  environmental  planning  and  bears  monitoring. 

Although  the  existing  literature  is  rather  limited,  the  future  literature  can  be  expected  to  blossom. 
During  the  preparation  of  this  report  a  number  of  literature  sources  that  could  be  worth  monitoring  in  the 
future,  in  order  to  remain  aware  of  new  developments  in  the  literature,  have  been  found.  They  are  described 
below  under  the  appropriate  heading. 

JOURNALS  AND  PROFESSIONAL  PUBLICATIONS 

Despite  the  availability  and  ease  of  use  of  online  search  facilities,  we  would  like  to  plug  an  old  tried 
and  true  method,  library  browsing.  Sometimes  the  latest  literature  does  not  turn  up  on  the  online  searches 
until  after  a  considerable  time  lag.  New  journals  may  take  quite  awhile  to  appear  among  the  periodicals 
reviewed  by  the  online  services.  Other  publications  may  be  overlooked  because  they  are  not  formal 
publications  available  for  purchase.  The  online  searches  failed  to  identify  quite  a  few  of  the  52  items 
selected  for  review  for  these  and  possibly  other  reasons. 

Analysts  interested  in  the  literature  should  from  time-to-time  visit  a  major  research  university  and 
spend  an  afternoon  browsing  the  periodicals  on  display.  Check  the  lists  of  periodicals  on  microfiche,  but 
be  sure  to  wander  among  the  journals  and  magazines  themselves.  It  takes  just  seconds  to  browse  through 
the  titles  to  see  what  kind  of  work  the  periodical  offers.  Nothing  gives  you  a  flavor  for  the  magazine  like 
flipping  through  it  and  observing  the  subject  matter,  the  difficulty  of  the  writing,  the  nature  of  the  advertisers, 
and  the  backgrounds  of  the  authors. 
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Following  is  a  list  of  some  periodicals  that  we  think  will  likely  provide  valuable  information  in  the 
future.  Many  of  them  may  be  too  expensive  for  individual  Corps  Districts  to  purchase  by  subscription,  and 
others  might  be  used  too  infrequently  to  justify  the  expense.  However,  periodic  monitoring  of  these  titles 
would  likely  be  worthwhile  for  future  planning  studies  and  research  efforts. 

Following  are  28  periodicals  that  would  be  useful  to  monitor  for  future  developments  in  risk  analysis 
related  to  the  environment.  The  five  periodicals  that  proved  most  useful  in  this  review  have  been  marked 
with  an  asterisk  (*). 

Challenge 

Conservation  Biology 

Ecodecision  Environment  and  Policy  Management 
Ecological  Bulletins 
Ecological  Economics'^ 

Ecological  Modeling 
Endangered  Species  Update 
Environment  and  Planning 
Environmental  Economics 
Environmental  Ethics 
The  Environmental  Forum 
The  Environmental  Professional'^ 

Environmental  Research 
Environmental  and  Resource  Economics 
Environmental  Science  and  Technology 
Environmental  Software'^ 

Global  Environmental  Change 

Journal  of  Agricultural  and  Resource  Economics 

Journal  of  Environmental  Economics  and  Management 

Journal  of  Environmental  Management 

Journal  of  Risk  and  Uncertainty 

Marine  Resource  Economics 

Natural  Resources  Journal 

Project  Appraisal 

Risk 

Risk  Analysis'^ 

Science  of  the  Total  Environment 
Water  Resources  Bulletin 
Water  Resources  Research 
Water  Science  and  Technology 
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ONLINE  SEARCHES 

There  is  nothing  like  sitting  down  to  a  computer,  typing  in  a  few  key  words,  and  generating  a  nice 
long  list  of  references.  When  it  works  it’s  a  wonderful  thing.  It  frequently  doesn’t  work,  however,  for  a 
variety  of  reasons.  Consider  some  of  the  online  library  searches  like  CARL.  These  services  allow  the  user 
to  access  the  card  files  of  selected  libraries  from  all  over  the  country.  The  libraries  you  can  search  depend 
on  the  libraries  that  participate  in  that  particular  service.  Frequently  there  will  be  a  service  connecting  most 
of  the  college  and  university  libraries  in  a  state. 

These  services  generally  list  only  book  titles,  not  journal  articles.  The  subject  matter  treated  in 
books  can  be  from  several  months  to  a  few  years  behind  the  material  covered  in  periodicals.  Many  colleges 
and  universities  make  it  a  policy  not  to  stock  books  that  are  required  for  course  work.  They  disappear  too 
frequently.  Excluding  course  texts  can  deprive  the  searcher  of  access  to  some  of  the  latest  and  best  titles. 
At  a  minimum,  these  books  are  often  best  suited  to  readers  seeking  to  learn  techniques. 

Another  common  limitation  of  these  services  is  that  it  is  often  impossible  to  print  the  search  results 
to  a  file  that  can  be  downloaded.  Often  printing  out  long  lists  of  citations  is  the  only  way  to  get  a  complete 
list  of  results. 

Finding  the  proper  keywords  can  be  more  art  than  science.  Searching  a  term  like  “risk”  yielded 
thousands  of  references  in  our  early  search.  It  was  impossible  to  review  each  of  these  citations  and  it  would 
have  been  wasteful  to  print  them  all.  The  problem  you  have  to  overcome  is  that  a  writer  or  reader  of  the 
book  had  to  guess  what  the  keywords  for  that  book  were.  You,  in  turn,  must  guess  what  keywords  they  used 
to  catalogue  their  work.  That  is  easier  said  than  done.  It  will  be  necessary  for  the  language  of  risk  analysis 
to  become  more  standardized  for  these  problems  to  be  avoided  in  the  future. 

In  addition  to  library  services  there  are  commercial  services  one  can  use.  Several  sell  their  entire 
set  of  citations  on  CD-ROM.  These  typically  start  at  about  $1,200  and  go  up.  Other  services  allow  users 
to  subscribe.  In  this  case  the  user  obtains  an  account  and  can  use  the  service  whenever  they  desire.  Libraries 
are  common  clients  of  such  subscription  services.  Still  other  services  will  conduct  searches  for  the  users  on 
a  case-by-case  basis.  Users  can  often  access  these  services  at  college/university  libraries. 

Expense  is  an  obvious  consideration.  Purchase  of  citation  databases  can  be  prohibitively  expensive. 
However,  they  may  be  reasonable  expenditures  for  an  organization  like  the  Corps  if  they  can  be  purchased 
and  maintained  at  a  central  location  available  to  all.  Another  constraint  is  creativity,  a  word  we  use  to  mean 
the  ability  to  get  what  is  of  interest  to  you  in  the  database  out  of  the  database.  This  is  the  keyword  problem 
again.  It  has  been  our  experience  that  there  is  no  substitute  for  a  patient,  knowledgeable  and  creative 
librarian  familiar  with  the  use  of  the  particular  online  database  for  getting  useful  information  out  of  the 
system. 


A  third  kind  of  online  service  is  that  offered  by  government  agencies.  EPA  has  several  of  these. 
They  are  not  usually  very  user  friendly  and  can  take  quite  a  bit  of  time  to  learn  to  use.  A  fourth  potential 
source  of  information,  not  used  in  this  research,  could  be  news  groups  on  the  Internet. 
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Following  are  a  few  of  the  specialty  online  services  that  should  be  considered  by  analysts.  Library 
online  services  are  not  explicitly  mentioned  because  they  are  ubiquitous. 

The  following  services  are  offered  for  general  information.  Relatively  few  of  them  were  used  during 
this  literature  review. 

Geographic  Information  Systems  Bulletin  Board  (GISNET) 

GISNET  is  an  online  PC-based  bulletin  board  system  from  EPA’s  National  GIS  Program. 

GISNET  3405R 
401  M  Street,  SW 
Washington,  D.C.  20460 
Phone:  703-557-3011 
Fax:  703-557-3186 

Office  of  Air  Quality  Planning  and  Standards  Technology  Transfer  Network  Bulletin  Board  System 
(OAQPS  TTN) 

The  OAQPS  TTN  is  a  network  of  electronic  bulletin  boards  that  provides  information  and 
technology  exchange  in  different  areas  of  air  pollution  control  ranging  from  emission  test  methods  to 
regulatory  air  quality  models. 

Office  of  Air  Quality  Planning  and  Standards 
Technology  Transfer  Network 
Bulletin  Board  System 
Research  Triangle  Park,  NC  2771 1 
Phone:  919-541-5384 

Clean-Up  Information  Bulletin  Board  System  (CLU-IN) 

The  CLU-IN  Bulletin  Board  offers  a  number  of  services  including  online  messages  and  bulletins; 
computer  files,  programs,  and  databases;  and  Special  Interest  Group  areas. 

U.S.  EPA 

Technology  Innovation  Office,  IS-1  lOW 
401  M  Street,  SW 
Washington,  DC  20460 
Phone:  301-589-8368  or  703-308-8827 
Fax:  301-589-8487 
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Office  of  Research  and  Development  Electronic  Bulletin  Board  System  (ORD  BBS) 

The  ORD  BBS  is  an  online,  text-searchable  database  of  every  ORD  publication  produced  since  1976, 
including  over  17,000  citations. 

U.S.  EPA 

Center  for  Environmental  Research  Information 
26  West  Martin  Luther  King  Drive 
Cincinnati,  OH  45268 
Phone:  513-569-7272 
Fax:  513-569-7700 

Information  Systems  Inventory  (ISI) 

ISI  at  EPA  is  an  automated  catalog  of  roughly  500  of  the  Agency’s  manual  and  electronic  data 
systems,  ranging  from  scientific  models  to  administrative  systems  to  environmental  databases. 

ISI  Administrator 

OIRM/IMSD 

3403 

U.S.  EPA 
401  M  Street,  SW 
Washington,  DC  20460 
Phone:  202-260-1077 

Environmental  Bibliography 

Environmental  Bibliography  provides  access  to  the  contents  of  periodicals  dealing  with  the 
environment  from  1973  to  the  present.  It  corresponds  to  the  Environmental  Periodicals  Bibliography  with 
over  500  journal  titles. 

Environmental  Studies  Institute 
International  Academy  at  Santa  Barbara 
800  Garden  Street,  Suite  D 
Santa  Barbara,  CA  93103 
Phone:  805-965-5010 
Fax:  805-965-6071 

Enviroline 

Enviroline  covers  the  world’s  environmental  related  information  from  1971 .  It  corresponds  to  the 
printed  Environment  Abstracts. 
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Environment  Abstracts 
Congressional  Information  Service,  Inc. 
4520  East-West  Highway,  Suite  800 
Bethesda,  MD  20814-3389 
Phone:  800-638-8380 
Fax:  301-657-3203 


Others 


There  are  a  number  of  additional  abstracts,  databases,  and  indices  that  may  warrant  monitoring  for 
future  developments  in  the  literature.  These  include  but  are  not  limited  to  the  following: 

Current  Law  Index 

Environment  Abstracts 

Environmental  Periodicals  Bibliography 

Health  and  Safety  Abstracts 

Index  to  Legal  Periodicals 

Journal  of  Economic  Literature 

Legaltrac 

Legal  Resources  Index 

PAIS  Print  and  Computer  Indices 

Pollution  Abstracts 

Risk  Abstracts 

UnCover  Company 

WESTLAW 


INTERNET  RESOURCES 

It  is  with  some  trepidation  that  we  offer  some  suggestions  for  monitoring  developments  in  this  field 
through  the  Internet.  There  is  no  question  about  the  value  or  volume  of  resources  to  be  found  on  the  Internet. 
It  is  a  gold  mine.  The  trepidation  stems  from  the  fact  that  this  information  is  likely  to  be  obsolete  and  will 
surely  be  incomplete  almost  as  soon  as  it  is  published.  The  most  important  message  of  this  section  is 
therefore  that  any  interested  environmental  planner  or  risk  analyst  should  regularly  monitor  the  resources 
of  the  Internet  for  developments  in  this  field. 

A  recent  search  for  risk-related  news  groups  failed  to  identify  any  at  this  time.  RISKANAL  is  a 
particularly  interesting  mailing  list  for  risk  analysts.  The  e-mail  address  for  administration  of  this  list  is 
“Hstserv@listserv.pnl.gov”.  The  list  is  maintained  by  the  Pacific  Northwest  National  Laboratory  of  the 
Battelle  Corporation  in  Richland,  Washington.  RISKANAL  is  a  non-moderated  list  devoted  to  risk  analysis 
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discussions  initiated  by  subscribers.  At  this  writing  there  are  about  600  subscribers  and  the  volume  of  mail 
averages  about  3-6  pieces  per  day. 

Web  sites  on  the  World  Wide  Web  (WWW)  are  currently  the  richest  source  of  risk  analysis 
information.  A  recent  search  of  the  WWW  using  the  words  “risk  analysis”  and  the  Web  Crawler  search 
engine  returned  approximately  1000  documents.  Suffice  it  to  say  there  are  a  great  number  of  potentially 
interesting  home  pages  on  the  WWW.  Be  forewarned,  the  quality  of  information  on  these  sites  is  variable. 
Some  sites  are  solicitations  for  business,  some  are  rarely  or  poorly  maintained  and  updated  (e.g.,  they 
disappear  or  change  address  without  warning),  and  others  seem  always  to  be  under  construction. 
Fortunately,  however,  there  are  many  excellent  sites  with  a  wealth  of  information.  Following  are  a  few 
examples  of  some  potentially  useful  web  sites; 

Risk  Home  Page:  http://www.fplc.edu/tfield/profRisk.html 

This  is  an  excellent  place  to  begin.  The  Risk  Home  Page  deals  primarily  with  health,  safety,  and  the  . 
environment.  It  includes  an  excellent  annotated  index  of  articles  from  the  journal  Risk  and  many  other 
interesting  features.  Other  examples  of  risk-related  web  sites  follow,  and  the  contents  vary  from  site-to-site. 

Decision/  Risk  Analysis:  http://www.lumina.com/DA/ 

Probabilistic  Risk  and  Hazard  Analysis  (TSA-1 1):  http://www.ta52.lanl.gov/tsal  l.html 

FERC  Risk  and  Environment:  http://www.oml.gov/ORNLReview/rev26-34/text/hydmain.html/ 

Environmental  Risk  Assessment  Projects:  http://fiwsys.lanl.gov/%7Bjks/TSA/BJK.html 

In  addition  to  these  risk-related  sites  it  can  be  beneficial  to  monitor  government  agency  sites.  One  need  only 
use  the  agency  name  with  a  WWW  search  engine  to  identify  sites  of  particular  interest.  The  IWR  home  page 
address  is: 

HTTP;//WWW.WRC-NDC.USACE.ARMY.MIL/IWR/INDEX.HTM 

PUBLISHERS 

During  this  literature  review  we  became  aware  of  a  publisher  that  seems  to  be  developing  a  specialty 
in  risk  analysis  relating  to  environmental  issues.  It  would  appear  to  be  worth  monitoring  the  offerings  of  this 
publisher. 

Lewis  Publishers 

2000  Corporate  Blvd.,  N.W. 

Boca  Raton,  FL  3343 1 
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